2024-04-25T00:54:02.592 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T00:54:02.592 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298 --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/orig.config.yaml 2024-04-25T00:54:02.638 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T00:54:02.726 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672298' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Reduced data availability - Degraded data redundancy sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0HvAv8M2hCk3E5ruY79aT3zBJNLSOaRGctm1CEq9ALA9tfcqfEXFdS8Yun7cainNS4pwa5ZFY1I3fea1XPbOM= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHaEzxCOKVFgS3Fm1PlIYxXr3G+iem+4DYLvPFbtwTmW0Xw0RffLfUajXYHhN6uHJtu88OK+vnx44TjqX91ZoW8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_acls teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T00:54:02.726 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T00:54:02.727 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T00:54:02.727 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T00:54:02.729 INFO:teuthology.task.internal:Checking packages... 2024-04-25T00:54:02.752 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T00:54:02.752 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T00:54:02.752 INFO:teuthology.packaging:ref: None 2024-04-25T00:54:02.752 INFO:teuthology.packaging:tag: None 2024-04-25T00:54:02.753 INFO:teuthology.packaging:branch: squid 2024-04-25T00:54:02.753 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:54:02.753 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-25T00:54:02.965 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T00:54:02.967 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T00:54:02.974 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T00:54:02.974 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T00:54:02.983 INFO:teuthology.task.internal:Saving configuration 2024-04-25T00:54:02.998 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T00:54:03.004 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T00:54:03.028 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:48:24.262291', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK0ZNgosqn+QK9H1LMjWvANXM8VJsfDL9QGbFIEhGWNf'} 2024-04-25T00:54:03.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 00:48:24.260763', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMrZdiC91SEsK0zIMo0ujdde2pimVj+Nz3GE3h43J33f'} 2024-04-25T00:54:03.048 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T00:54:03.054 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T00:54:03.054 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T00:54:03.054 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T00:54:03.122 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb5c6f0bee0>, signals=[15]) 2024-04-25T00:54:03.123 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T00:54:03.128 INFO:teuthology.task.internal:Opening connections... 2024-04-25T00:54:03.128 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T00:54:03.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:54:03.214 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T00:54:03.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:54:03.293 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T00:54:03.300 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-25T00:54:03.305 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-25T00:54:03.306 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-25T00:54:03.356 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:54:03.356 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="Ubuntu" 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="22.04" 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_CODENAME=jammy 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:ID=ubuntu 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE=debian 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:54:03.357 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:54:03.358 INFO:teuthology.orchestra.run.smithi033.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:54:03.358 INFO:teuthology.orchestra.run.smithi033.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:54:03.359 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-25T00:54:03.392 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-25T00:54:03.397 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-25T00:54:03.398 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="Ubuntu" 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="22.04" 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_CODENAME=jammy 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:ID=ubuntu 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE=debian 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T00:54:03.449 INFO:teuthology.orchestra.run.smithi138.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T00:54:03.450 INFO:teuthology.orchestra.run.smithi138.stdout:UBUNTU_CODENAME=jammy 2024-04-25T00:54:03.450 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-25T00:54:03.490 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T00:54:03.499 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T00:54:03.505 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T00:54:03.505 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:54:03.508 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-25T00:54:03.512 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T00:54:03.517 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T00:54:03.518 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:54:03.556 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-25T00:54:03.572 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T00:54:03.623 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T00:54:03.638 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T00:54:03.638 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T00:54:03.639 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:54:03.639 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T00:54:03.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:03.644 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-04-25T00:54:03.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:54:03.645 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-25T00:54:03.691 INFO:teuthology.orchestra.run.smithi033.stdout:5.15.0-56-generic 2024-04-25T00:54:03.692 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.15.0-56-generic 2024-04-25T00:54:03.692 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2024-04-25T00:54:03.693 INFO:teuthology.orchestra.run.smithi138.stdout:5.15.0-56-generic 2024-04-25T00:54:03.693 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.15.0-56-generic 2024-04-25T00:54:03.694 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get clean 2024-04-25T00:54:03.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-04-25T00:54:03.825 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get update 2024-04-25T00:54:04.022 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:54:04.087 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:54:04.120 INFO:teuthology.orchestra.run.smithi138.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T00:54:04.125 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:54:04.154 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:54:04.182 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T00:54:04.306 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:54:04.520 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:54:04.551 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T00:54:04.553 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:54:04.557 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T00:54:04.570 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:54:04.573 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:54:04.573 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:54:04.608 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:54:04.608 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:54:04.612 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:54:04.612 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:54:04.622 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:54:04.628 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:54:04.630 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:54:04.630 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:54:04.657 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:54:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:54:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:54:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:54:04.733 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:54:04.889 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T00:54:04.972 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T00:54:04.990 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T00:54:04.994 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [618 kB] 2024-04-25T00:54:04.999 INFO:teuthology.orchestra.run.smithi138.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T00:54:05.000 INFO:teuthology.orchestra.run.smithi138.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T00:54:05.001 INFO:teuthology.orchestra.run.smithi138.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T00:54:05.006 INFO:teuthology.orchestra.run.smithi138.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:54:05.021 INFO:teuthology.orchestra.run.smithi138.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:54:05.022 INFO:teuthology.orchestra.run.smithi138.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:54:05.033 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T00:54:05.048 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T00:54:05.048 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T00:54:05.084 INFO:teuthology.orchestra.run.smithi138.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:54:05.085 INFO:teuthology.orchestra.run.smithi138.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:54:05.092 INFO:teuthology.orchestra.run.smithi138.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:54:05.096 INFO:teuthology.orchestra.run.smithi138.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T00:54:05.110 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T00:54:05.111 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T00:54:05.119 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:54:05.127 INFO:teuthology.orchestra.run.smithi138.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T00:54:05.127 INFO:teuthology.orchestra.run.smithi138.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T00:54:05.135 INFO:teuthology.orchestra.run.smithi138.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T00:54:05.256 INFO:teuthology.orchestra.run.smithi138.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:54:05.283 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T00:54:05.311 INFO:teuthology.orchestra.run.smithi138.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T00:54:05.716 INFO:teuthology.orchestra.run.smithi138.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T00:54:05.721 INFO:teuthology.orchestra.run.smithi138.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T00:54:05.736 INFO:teuthology.orchestra.run.smithi138.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T00:54:05.736 INFO:teuthology.orchestra.run.smithi138.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T00:54:05.738 INFO:teuthology.orchestra.run.smithi138.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T00:54:05.839 INFO:teuthology.orchestra.run.smithi138.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:54:05.865 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T00:54:05.950 INFO:teuthology.orchestra.run.smithi138.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:54:05.970 INFO:teuthology.orchestra.run.smithi138.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:54:05.971 INFO:teuthology.orchestra.run.smithi138.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:54:05.972 INFO:teuthology.orchestra.run.smithi138.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:54:05.977 INFO:teuthology.orchestra.run.smithi138.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:54:05.978 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T00:54:05.978 INFO:teuthology.orchestra.run.smithi138.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:54:05.978 INFO:teuthology.orchestra.run.smithi138.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:54:05.983 INFO:teuthology.orchestra.run.smithi138.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:54:05.988 INFO:teuthology.orchestra.run.smithi138.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:54:05.996 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T00:54:05.997 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T00:54:05.998 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T00:54:06.001 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T00:54:06.002 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T00:54:06.002 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T00:54:06.008 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T00:54:06.014 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T00:54:06.154 INFO:teuthology.orchestra.run.smithi138.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:54:06.178 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T00:54:06.237 INFO:teuthology.orchestra.run.smithi138.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:54:06.260 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:54:06.321 INFO:teuthology.orchestra.run.smithi138.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T00:54:06.373 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T00:54:06.403 INFO:teuthology.orchestra.run.smithi138.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:54:06.431 INFO:teuthology.orchestra.run.smithi033.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T00:54:06.435 INFO:teuthology.orchestra.run.smithi138.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:54:06.465 INFO:teuthology.orchestra.run.smithi033.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T00:54:10.816 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 13.3 MB in 3s (5,231 kB/s) 2024-04-25T00:54:10.973 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 13.3 MB in 3s (5,188 kB/s) 2024-04-25T00:54:12.181 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T00:54:12.200 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:54:12.251 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T00:54:12.408 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T00:54:12.425 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:54:12.430 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T00:54:12.431 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T00:54:12.478 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T00:54:12.587 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:54:12.587 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:54:12.587 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:54:12.588 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:54:12.588 INFO:teuthology.orchestra.run.smithi138.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:54:12.588 INFO:teuthology.orchestra.run.smithi138.stdout: python2.7-minimal 2024-04-25T00:54:12.588 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:54:12.589 INFO:teuthology.orchestra.run.smithi138.stdout:The following additional packages will be installed: 2024-04-25T00:54:12.589 INFO:teuthology.orchestra.run.smithi138.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:54:12.589 INFO:teuthology.orchestra.run.smithi138.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:54:12.589 INFO:teuthology.orchestra.run.smithi138.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:54:12.590 INFO:teuthology.orchestra.run.smithi138.stdout:Suggested packages: 2024-04-25T00:54:12.590 INFO:teuthology.orchestra.run.smithi138.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:54:12.591 INFO:teuthology.orchestra.run.smithi138.stdout:Recommended packages: 2024-04-25T00:54:12.591 INFO:teuthology.orchestra.run.smithi138.stdout: thermald 2024-04-25T00:54:12.643 INFO:teuthology.orchestra.run.smithi138.stdout:The following NEW packages will be installed: 2024-04-25T00:54:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:54:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:54:12.643 INFO:teuthology.orchestra.run.smithi138.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:54:12.644 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages will be upgraded: 2024-04-25T00:54:12.645 INFO:teuthology.orchestra.run.smithi138.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:54:12.659 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-04-25T00:54:12.660 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-04-25T00:54:12.824 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:54:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:54:12.824 INFO:teuthology.orchestra.run.smithi033.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:54:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:54:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T00:54:12.825 INFO:teuthology.orchestra.run.smithi033.stdout: python2.7-minimal 2024-04-25T00:54:12.825 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:54:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:54:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:Need to get 113 MB of archives. 2024-04-25T00:54:12.825 INFO:teuthology.orchestra.run.smithi138.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:54:12.826 INFO:teuthology.orchestra.run.smithi138.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:54:12.826 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-04-25T00:54:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:54:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T00:54:12.826 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T00:54:12.827 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-04-25T00:54:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T00:54:12.828 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2024-04-25T00:54:12.828 INFO:teuthology.orchestra.run.smithi033.stdout: thermald 2024-04-25T00:54:12.864 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-04-25T00:54:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T00:54:12.864 INFO:teuthology.orchestra.run.smithi033.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T00:54:12.865 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T00:54:12.866 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2024-04-25T00:54:12.866 INFO:teuthology.orchestra.run.smithi033.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T00:54:13.046 INFO:teuthology.orchestra.run.smithi033.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:54:13.046 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 113 MB of archives. 2024-04-25T00:54:13.046 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T00:54:13.046 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T00:54:14.252 INFO:teuthology.orchestra.run.smithi138.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:54:14.450 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T00:54:14.603 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:54:14.786 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T00:54:16.830 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:54:16.830 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:54:16.831 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:54:16.931 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T00:54:16.931 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T00:54:16.932 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T00:54:17.730 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:54:17.812 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:54:17.990 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T00:54:18.082 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T00:54:18.220 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 113 MB in 5s (22.0 MB/s) 2024-04-25T00:54:18.404 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:54:18.491 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 113 MB in 5s (21.8 MB/s) 2024-04-25T00:54:18.670 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T00:54:20.458 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:54:20.462 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:20.568 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:20.697 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T00:54:20.701 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:20.785 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:23.153 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:54:23.174 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:23.212 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T00:54:23.234 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:23.267 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:23.310 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:23.764 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:54:23.785 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:23.822 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:23.824 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T00:54:23.845 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:23.882 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:30.295 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:54:30.437 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:54:30.438 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:54:30.558 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:54:30.769 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:54:30.890 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:54:30.924 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:54:31.052 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:54:31.187 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:54:31.210 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:54:31.245 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:54:31.331 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T00:54:31.353 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T00:54:31.414 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:54:36.950 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:54:36.988 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:37.050 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:37.345 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T00:54:37.383 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T00:54:37.420 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:40.615 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:54:40.727 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:54:41.077 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:54:41.136 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T00:54:41.190 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:41.259 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T00:54:41.316 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:54:41.419 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:41.626 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T00:54:41.723 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:41.824 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T00:54:41.977 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:42.769 INFO:teuthology.orchestra.run.smithi033.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:54:42.770 INFO:teuthology.orchestra.run.smithi033.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:54:42.770 INFO:teuthology.orchestra.run.smithi033.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:54:42.770 INFO:teuthology.orchestra.run.smithi033.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:54:42.929 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:43.312 INFO:teuthology.orchestra.run.smithi138.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:54:43.312 INFO:teuthology.orchestra.run.smithi138.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T00:54:43.312 INFO:teuthology.orchestra.run.smithi138.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:54:43.313 INFO:teuthology.orchestra.run.smithi138.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T00:54:43.488 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:44.357 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:44.849 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:46.074 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:54:46.225 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:54:46.351 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:46.418 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:54:46.419 INFO:teuthology.orchestra.run.smithi033.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:54:46.450 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T00:54:46.592 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T00:54:46.735 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T00:54:46.794 INFO:teuthology.orchestra.run.smithi138.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T00:54:46.794 INFO:teuthology.orchestra.run.smithi138.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:55:00.766 INFO:teuthology.orchestra.run.smithi033.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:55:00.766 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:55:00.787 INFO:teuthology.orchestra.run.smithi033.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:55:00.811 INFO:teuthology.orchestra.run.smithi033.stdout:Generating grub configuration file ... 2024-04-25T00:55:00.953 INFO:teuthology.orchestra.run.smithi138.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T00:55:00.954 INFO:teuthology.orchestra.run.smithi138.stdout:Sourcing file `/etc/default/grub' 2024-04-25T00:55:00.979 INFO:teuthology.orchestra.run.smithi138.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:55:00.998 INFO:teuthology.orchestra.run.smithi138.stdout:Generating grub configuration file ... 2024-04-25T00:55:01.059 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:55:01.088 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:55:01.321 INFO:teuthology.orchestra.run.smithi138.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:55:01.362 INFO:teuthology.orchestra.run.smithi138.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:55:01.366 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:55:01.372 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:55:01.445 INFO:teuthology.orchestra.run.smithi033.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:55:01.451 INFO:teuthology.orchestra.run.smithi033.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:55:01.605 INFO:teuthology.orchestra.run.smithi033.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:55:01.606 INFO:teuthology.orchestra.run.smithi033.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:55:01.606 INFO:teuthology.orchestra.run.smithi033.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:55:01.649 INFO:teuthology.orchestra.run.smithi033.stdout:done 2024-04-25T00:55:01.963 INFO:teuthology.orchestra.run.smithi138.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:55:01.969 INFO:teuthology.orchestra.run.smithi138.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:55:02.030 INFO:teuthology.orchestra.run.smithi138.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:55:02.036 INFO:teuthology.orchestra.run.smithi138.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:55:02.174 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2024-04-25T00:55:02.195 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 21 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.15.0.105.102 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:55:02.196 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2024-04-25T00:55:02.197 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:55:02.197 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:55:02.197 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:55:02.197 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T00:55:02.197 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:55:02.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get clean 2024-04-25T00:55:02.202 INFO:teuthology.orchestra.run.smithi138.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:55:02.203 INFO:teuthology.orchestra.run.smithi138.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:55:02.203 INFO:teuthology.orchestra.run.smithi138.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:55:02.255 INFO:teuthology.orchestra.run.smithi138.stdout:done 2024-04-25T00:55:02.364 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-04-25T00:55:02.517 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:55:02.517 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:55:02.524 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:55:02.544 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:55:02.768 DEBUG:teuthology.orchestra.run.smithi138:> dpkg -s linux-image-generic 2024-04-25T00:55:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:Package: linux-image-generic 2024-04-25T00:55:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:Status: install ok installed 2024-04-25T00:55:02.787 INFO:teuthology.orchestra.run.smithi138.stdout:Priority: optional 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Section: kernel 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Installed-Size: 21 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: amd64 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Source: linux-meta 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Version: 5.15.0.105.102 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Recommends: thermald 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout:Description: Generic Linux kernel image 2024-04-25T00:55:02.788 INFO:teuthology.orchestra.run.smithi138.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:55:02.789 INFO:teuthology.orchestra.run.smithi138.stdout: available. 2024-04-25T00:55:02.789 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T00:55:02.789 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T00:55:02.789 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T00:55:02.789 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T00:55:02.789 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T00:55:02.789 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get clean 2024-04-25T00:55:02.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get update 2024-04-25T00:55:03.149 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T00:55:03.149 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T00:55:03.199 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T00:55:03.256 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T00:55:03.953 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T00:55:03.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:55:04.036 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T00:55:04.239 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-04-25T00:55:04.240 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-04-25T00:55:04.426 INFO:teuthology.orchestra.run.smithi033.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:55:04.426 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:55:04.426 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:55:04.426 INFO:teuthology.orchestra.run.smithi033.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:55:04.427 INFO:teuthology.orchestra.run.smithi033.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:55:04.427 INFO:teuthology.orchestra.run.smithi033.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:55:04.427 INFO:teuthology.orchestra.run.smithi033.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:55:04.427 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:55:04.468 INFO:teuthology.orchestra.run.smithi033.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:55:04.470 DEBUG:teuthology.orchestra.run.smithi033:> dpkg -s linux-image-generic 2024-04-25T00:55:04.486 INFO:teuthology.orchestra.run.smithi033.stdout:Package: linux-image-generic 2024-04-25T00:55:04.486 INFO:teuthology.orchestra.run.smithi033.stdout:Status: install ok installed 2024-04-25T00:55:04.486 INFO:teuthology.orchestra.run.smithi033.stdout:Priority: optional 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Section: kernel 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Installed-Size: 21 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: amd64 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Source: linux-meta 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Version: 5.15.0.105.102 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Recommends: thermald 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout:Description: Generic Linux kernel image 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:55:04.487 INFO:teuthology.orchestra.run.smithi033.stdout: available. 2024-04-25T00:55:04.488 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-04-25T00:55:04.518 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T00:55:04.535 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T00:55:04.538 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.5brm2QWM1o 2024-04-25T00:55:04.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub/grub.cfg /tmp/tmp.5brm2QWM1o 2024-04-25T00:55:04.586 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T00:55:04.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.5brm2QWM1o 2024-04-25T00:55:04.714 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.5brm2QWM1o is 10KB 2024-04-25T00:55:04.727 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.5brm2QWM1o 2024-04-25T00:55:04.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:55:04.764 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T00:55:04.766 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T00:55:04.790 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T00:55:04.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:55:04.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:55:04.858 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:55:04.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo update-grub 2024-04-25T00:55:04.926 INFO:teuthology.orchestra.run.smithi138.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T00:55:04.926 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T00:55:04.926 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T00:55:04.927 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T00:55:04.927 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T00:55:04.927 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T00:55:04.927 INFO:teuthology.orchestra.run.smithi138.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T00:55:04.927 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T00:55:04.964 INFO:teuthology.orchestra.run.smithi138.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T00:55:04.966 DEBUG:teuthology.orchestra.run.smithi138:> dpkg -s linux-image-generic 2024-04-25T00:55:05.027 INFO:teuthology.orchestra.run.smithi138.stdout:Package: linux-image-generic 2024-04-25T00:55:05.027 INFO:teuthology.orchestra.run.smithi138.stdout:Status: install ok installed 2024-04-25T00:55:05.027 INFO:teuthology.orchestra.run.smithi138.stdout:Priority: optional 2024-04-25T00:55:05.027 INFO:teuthology.orchestra.run.smithi138.stdout:Section: kernel 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Installed-Size: 21 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: amd64 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Source: linux-meta 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Version: 5.15.0.105.102 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Recommends: thermald 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout:Description: Generic Linux kernel image 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout: This package will always depend on the latest generic kernel image 2024-04-25T00:55:05.028 INFO:teuthology.orchestra.run.smithi138.stdout: available. 2024-04-25T00:55:05.029 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-25T00:55:05.072 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.iEWE0sVeD2 2024-04-25T00:55:05.072 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub/grub.cfg /tmp/tmp.iEWE0sVeD2 2024-04-25T00:55:05.138 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.iEWE0sVeD2 2024-04-25T00:55:05.250 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.iEWE0sVeD2 is 10KB 2024-04-25T00:55:05.263 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.iEWE0sVeD2 2024-04-25T00:55:05.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T00:55:05.324 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T00:55:05.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T00:55:05.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T00:55:05.402 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T00:55:05.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo update-grub 2024-04-25T00:55:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:55:06.116 INFO:teuthology.orchestra.run.smithi033.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:55:06.118 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-04-25T00:55:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:55:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:55:06.535 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:55:06.541 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:55:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:55:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:55:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:55:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:55:06.706 INFO:teuthology.orchestra.run.smithi033.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:55:06.730 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-04-25T00:55:06.735 INFO:teuthology.orchestra.run.smithi138.stderr:Sourcing file `/etc/default/grub' 2024-04-25T00:55:06.736 INFO:teuthology.orchestra.run.smithi138.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T00:55:06.739 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-04-25T00:55:06.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-04-25T00:55:07.007 INFO:teuthology.orchestra.run.smithi138.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T00:55:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T00:55:07.355 INFO:teuthology.orchestra.run.smithi138.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T00:55:07.360 INFO:teuthology.orchestra.run.smithi138.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T00:55:07.402 INFO:teuthology.orchestra.run.smithi138.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T00:55:07.407 INFO:teuthology.orchestra.run.smithi138.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T00:55:07.528 INFO:teuthology.orchestra.run.smithi138.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T00:55:07.528 INFO:teuthology.orchestra.run.smithi138.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T00:55:07.528 INFO:teuthology.orchestra.run.smithi138.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T00:55:07.539 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-04-25T00:55:07.549 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-04-25T00:55:36.770 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T00:55:36.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-25T00:55:36.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:55:37.596 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T00:55:37.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T00:55:37.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:56:36.831 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T00:56:37.600 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T00:56:45.840 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-25T00:56:45.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:56:46.601 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T00:56:46.602 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:56:46.604 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-04-25T00:56:46.912 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-25T00:56:47.851 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-25T00:56:47.851 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T00:56:47.851 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-04-25T00:56:47.900 INFO:teuthology.orchestra.run.smithi033.stdout:5.15.0-105-generic 2024-04-25T00:56:47.901 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:56:47.901 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:56:47.901 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:56:48.901 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T00:56:48.902 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:56:48.983 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-04-25T00:56:49.001 DEBUG:teuthology.parallel:result is None 2024-04-25T00:56:58.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T00:56:58.605 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:56:58.876 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T00:56:59.467 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T00:56:59.468 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T00:56:59.468 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-25T00:56:59.514 INFO:teuthology.orchestra.run.smithi138.stdout:5.15.0-105-generic 2024-04-25T00:56:59.514 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T00:56:59.514 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T00:56:59.514 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T00:57:00.515 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T00:57:00.515 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T00:57:00.863 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-04-25T00:57:00.889 DEBUG:teuthology.parallel:result is None 2024-04-25T00:57:00.889 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T00:57:01.135 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T00:57:01.135 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:57:01.138 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T00:57:01.144 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T00:57:01.159 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T00:57:01.193 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T00:57:01.193 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:57:01.195 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T00:57:01.218 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T00:57:01.226 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T00:57:01.226 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:57:01.248 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T00:57:01.266 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:57:01.276 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:57:01.279 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:57:01.286 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T00:57:01.288 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T00:57:01.297 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T00:57:01.297 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:57:01.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T00:57:01.342 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T00:57:01.353 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T00:57:01.353 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:57:01.380 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T00:57:01.390 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:57:01.431 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:57:01.480 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T00:57:01.480 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:57:01.537 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T00:57:01.543 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T00:57:01.590 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T00:57:01.590 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T00:57:01.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-04-25T00:57:01.659 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-25T00:57:01.750 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T00:57:01.767 INFO:teuthology.task.internal:Starting timer... 2024-04-25T00:57:01.767 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T00:57:01.783 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T00:57:01.796 DEBUG:teuthology.task.selinux:Excluding smithi033: OS 'ubuntu' does not support SELinux 2024-04-25T00:57:01.796 DEBUG:teuthology.task.selinux:Excluding smithi138: OS 'ubuntu' does not support SELinux 2024-04-25T00:57:01.797 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T00:57:01.797 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T00:57:01.797 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T00:57:01.797 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T00:57:01.811 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T00:57:01.816 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T00:57:01.944 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T00:57:01.960 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T00:57:01.961 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T01:04:11.051 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-25T01:04:11.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-25T01:04:11.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:04:11.165 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-25T01:04:11.242 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-25T01:04:11.242 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T01:04:11.243 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:04:11.322 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T01:04:11.398 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T01:04:11.398 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T01:04:11.409 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T01:04:11.409 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:04:11.410 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:04:11.412 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:04:11.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Command line: ntpd -gq 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: ---------------------------------------------------- 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: corporation. Support and training for ntp-4 are 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: available at https://www.nwtime.org/support 2024-04-25T01:04:11.443 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: ---------------------------------------------------- 2024-04-25T01:04:11.444 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: proto: precision = 0.063 usec (-24) 2024-04-25T01:04:11.444 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: basedate set to 2022-02-04 2024-04-25T01:04:11.444 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: gps base set to 2022-02-06 (week 2196) 2024-04-25T01:04:11.444 INFO:teuthology.orchestra.run.smithi033.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T01:04:11.445 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T01:04:11.445 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T01:04:11.445 INFO:teuthology.orchestra.run.smithi033.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T01:04:11.446 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T01:04:11.446 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T01:04:11.446 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T01:04:11.446 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Listen normally on 3 ens1f0 172.21.15.33:123 2024-04-25T01:04:11.446 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Listen normally on 4 lo [::1]:123 2024-04-25T01:04:11.446 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:d294%4]:123 2024-04-25T01:04:11.446 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:11 ntpd[17156]: Listening on routing socket on fd #22 for interface updates 2024-04-25T01:04:11.466 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T01:04:11.466 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Command line: ntpd -gq 2024-04-25T01:04:11.466 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: ---------------------------------------------------- 2024-04-25T01:04:11.466 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T01:04:11.467 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T01:04:11.467 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: corporation. Support and training for ntp-4 are 2024-04-25T01:04:11.467 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: available at https://www.nwtime.org/support 2024-04-25T01:04:11.467 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: ---------------------------------------------------- 2024-04-25T01:04:11.467 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: proto: precision = 0.046 usec (-24) 2024-04-25T01:04:11.468 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: basedate set to 2022-02-04 2024-04-25T01:04:11.468 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: gps base set to 2022-02-06 (week 2196) 2024-04-25T01:04:11.468 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T01:04:11.468 INFO:teuthology.orchestra.run.smithi138.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T01:04:11.468 INFO:teuthology.orchestra.run.smithi138.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T01:04:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T01:04:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T01:04:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T01:04:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T01:04:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Listen normally on 3 enp3s0f1 172.21.15.138:123 2024-04-25T01:04:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Listen normally on 4 lo [::1]:123 2024-04-25T01:04:11.469 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f71%5]:123 2024-04-25T01:04:11.470 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:11 ntpd[17236]: Listening on routing socket on fd #22 for interface updates 2024-04-25T01:04:12.443 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:12 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:12.443 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:12 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:13.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:13.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:13.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:13.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:13.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:13.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:13.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:13 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:15 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:15.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:15.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:15.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:15.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:15.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:15.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:15 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:16.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:16 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:16.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:16 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:16.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:16 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:16.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:16 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:16.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:16 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:16.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:16 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:17.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:17.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:17 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:17.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:17.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:17.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:17.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:17.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:17.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:17.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:17.470 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:17 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:18 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:18 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:18 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:18 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:18 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:18 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:19.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:19 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.467 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stdout:25 Apr 01:04:19 ntpd[17236]: ntpd: time slew -0.001285 s 2024-04-25T01:04:19.468 INFO:teuthology.orchestra.run.smithi138.stdout:ntpd: time slew -0.001285s 2024-04-25T01:04:19.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T01:04:19.469 INFO:teuthology.orchestra.run.smithi138.stderr:25 Apr 01:04:19 ntpd[17236]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T01:04:19.519 INFO:teuthology.orchestra.run.smithi138.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T01:04:19.519 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================== 2024-04-25T01:04:19.519 INFO:teuthology.orchestra.run.smithi138.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:19.519 INFO:teuthology.orchestra.run.smithi138.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:19.519 INFO:teuthology.orchestra.run.smithi138.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:19.520 INFO:teuthology.orchestra.run.smithi138.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:20 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:20 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:20 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:20 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:20 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:20 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stdout:25 Apr 01:04:21 ntpd[17156]: ntpd: time slew -0.000757 s 2024-04-25T01:04:21.444 INFO:teuthology.orchestra.run.smithi033.stdout:ntpd: time slew -0.000757s 2024-04-25T01:04:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T01:04:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:25 Apr 01:04:21 ntpd[17156]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T01:04:21.494 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T01:04:21.495 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2024-04-25T01:04:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:21.495 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:04:21.496 INFO:teuthology.run_tasks:Running task install... 2024-04-25T01:04:21.505 DEBUG:teuthology.task.install:project ceph 2024-04-25T01:04:21.505 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T01:04:21.505 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T01:04:21.505 INFO:teuthology.task.install:Using flavor: default 2024-04-25T01:04:21.511 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T01:04:21.511 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T01:04:21.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-key list | grep Ceph 2024-04-25T01:04:21.512 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-key list | grep Ceph 2024-04-25T01:04:21.577 INFO:teuthology.orchestra.run.smithi138.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T01:04:21.594 INFO:teuthology.orchestra.run.smithi033.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T01:04:21.605 INFO:teuthology.orchestra.run.smithi138.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T01:04:21.606 INFO:teuthology.orchestra.run.smithi138.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T01:04:21.606 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T01:04:21.607 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T01:04:21.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:04:21.620 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T01:04:21.620 INFO:teuthology.orchestra.run.smithi033.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T01:04:21.621 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T01:04:21.621 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T01:04:21.621 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:04:21.745 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T01:04:21.745 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:04:21.761 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default/ 2024-04-25T01:04:21.761 INFO:teuthology.task.install.deb:Package version is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:04:21.857 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:04:21.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T01:04:21.872 DEBUG:teuthology.orchestra.run.smithi138:> sudo apt-get update 2024-04-25T01:04:21.936 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:04:21.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T01:04:21.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get update 2024-04-25T01:04:22.040 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T01:04:22.040 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T01:04:22.054 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T01:04:22.075 INFO:teuthology.orchestra.run.smithi138.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T01:04:22.128 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T01:04:22.128 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T01:04:22.141 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T01:04:22.148 INFO:teuthology.orchestra.run.smithi138.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T01:04:22.162 INFO:teuthology.orchestra.run.smithi033.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T01:04:22.185 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T01:04:22.222 INFO:teuthology.orchestra.run.smithi138.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T01:04:22.236 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T01:04:22.259 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T01:04:22.273 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T01:04:22.303 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T01:04:22.310 INFO:teuthology.orchestra.run.smithi033.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T01:04:22.347 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T01:04:22.404 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 28.9 kB in 0s (81.1 kB/s) 2024-04-25T01:04:22.424 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,410 B] 2024-04-25T01:04:22.508 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 28.9 kB in 0s (79.4 kB/s) 2024-04-25T01:04:23.381 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T01:04:23.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:04:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T01:04:23.485 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T01:04:23.500 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2445-gd2c45d97-1jammy cephadm=19.0.0-2445-gd2c45d97-1jammy ceph-mds=19.0.0-2445-gd2c45d97-1jammy ceph-mgr=19.0.0-2445-gd2c45d97-1jammy ceph-common=19.0.0-2445-gd2c45d97-1jammy ceph-fuse=19.0.0-2445-gd2c45d97-1jammy ceph-test=19.0.0-2445-gd2c45d97-1jammy ceph-volume=19.0.0-2445-gd2c45d97-1jammy radosgw=19.0.0-2445-gd2c45d97-1jammy python3-rados=19.0.0-2445-gd2c45d97-1jammy python3-rgw=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy python3-rbd=19.0.0-2445-gd2c45d97-1jammy libcephfs2=19.0.0-2445-gd2c45d97-1jammy libcephfs-dev=19.0.0-2445-gd2c45d97-1jammy librados2=19.0.0-2445-gd2c45d97-1jammy librbd1=19.0.0-2445-gd2c45d97-1jammy rbd-fuse=19.0.0-2445-gd2c45d97-1jammy python3-cephfs=19.0.0-2445-gd2c45d97-1jammy cephfs-shell=19.0.0-2445-gd2c45d97-1jammy cephfs-top=19.0.0-2445-gd2c45d97-1jammy cephfs-mirror=19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:04:23.561 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T01:04:23.656 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T01:04:23.656 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T01:04:23.766 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-04-25T01:04:23.766 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-04-25T01:04:23.845 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T01:04:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T01:04:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T01:04:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T01:04:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T01:04:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T01:04:23.847 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T01:04:23.847 INFO:teuthology.orchestra.run.smithi138.stdout:The following additional packages will be installed: 2024-04-25T01:04:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T01:04:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T01:04:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T01:04:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T01:04:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T01:04:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T01:04:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T01:04:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T01:04:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T01:04:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T01:04:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout:Suggested packages: 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout:Recommended packages: 2024-04-25T01:04:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: btrfs-tools 2024-04-25T01:04:23.957 INFO:teuthology.orchestra.run.smithi138.stdout:The following NEW packages will be installed: 2024-04-25T01:04:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T01:04:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T01:04:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T01:04:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T01:04:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T01:04:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T01:04:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T01:04:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T01:04:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T01:04:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T01:04:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T01:04:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T01:04:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T01:04:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T01:04:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T01:04:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: socat unzip xmlstarlet zip 2024-04-25T01:04:23.969 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T01:04:23.969 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T01:04:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T01:04:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T01:04:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T01:04:23.970 INFO:teuthology.orchestra.run.smithi033.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T01:04:23.971 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T01:04:23.973 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-04-25T01:04:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T01:04:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T01:04:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T01:04:23.973 INFO:teuthology.orchestra.run.smithi033.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T01:04:23.974 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-04-25T01:04:23.975 INFO:teuthology.orchestra.run.smithi033.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout:Recommended packages: 2024-04-25T01:04:23.976 INFO:teuthology.orchestra.run.smithi033.stdout: btrfs-tools 2024-04-25T01:04:24.086 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-04-25T01:04:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T01:04:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T01:04:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T01:04:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T01:04:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T01:04:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T01:04:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T01:04:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T01:04:24.089 INFO:teuthology.orchestra.run.smithi033.stdout: socat unzip xmlstarlet zip 2024-04-25T01:04:24.146 INFO:teuthology.orchestra.run.smithi138.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T01:04:24.146 INFO:teuthology.orchestra.run.smithi138.stdout:Need to get 180 MB of archives. 2024-04-25T01:04:24.146 INFO:teuthology.orchestra.run.smithi138.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T01:04:24.146 INFO:teuthology.orchestra.run.smithi138.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T01:04:24.154 INFO:teuthology.orchestra.run.smithi033.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T01:04:24.154 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 180 MB of archives. 2024-04-25T01:04:24.154 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 823 MB of additional disk space will be used. 2024-04-25T01:04:24.154 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T01:04:24.174 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T01:04:24.177 INFO:teuthology.orchestra.run.smithi138.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T01:04:24.186 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T01:04:24.228 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T01:04:24.235 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T01:04:24.259 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T01:04:24.263 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T01:04:24.273 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T01:04:24.276 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T01:04:24.309 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2445-gd2c45d97-1jammy [3,540 kB] 2024-04-25T01:04:24.330 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T01:04:24.334 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T01:04:24.346 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T01:04:24.350 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T01:04:24.351 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T01:04:24.359 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T01:04:24.361 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T01:04:24.364 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T01:04:24.365 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T01:04:24.365 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T01:04:24.387 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T01:04:24.390 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T01:04:24.391 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T01:04:24.392 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T01:04:24.393 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T01:04:24.411 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T01:04:24.411 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T01:04:24.411 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T01:04:24.412 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T01:04:24.412 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T01:04:24.432 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T01:04:24.432 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T01:04:24.436 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T01:04:24.437 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T01:04:24.437 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T01:04:24.452 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T01:04:24.453 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T01:04:24.454 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T01:04:24.454 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T01:04:24.456 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T01:04:24.472 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T01:04:24.473 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T01:04:24.477 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T01:04:24.480 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T01:04:24.481 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T01:04:24.482 INFO:teuthology.orchestra.run.smithi033.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T01:04:24.492 INFO:teuthology.orchestra.run.smithi033.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T01:04:24.494 INFO:teuthology.orchestra.run.smithi033.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T01:04:24.497 INFO:teuthology.orchestra.run.smithi033.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T01:04:24.498 INFO:teuthology.orchestra.run.smithi033.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T01:04:24.500 INFO:teuthology.orchestra.run.smithi033.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T01:04:24.513 INFO:teuthology.orchestra.run.smithi033.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T01:04:24.516 INFO:teuthology.orchestra.run.smithi033.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T01:04:24.517 INFO:teuthology.orchestra.run.smithi033.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T01:04:24.518 INFO:teuthology.orchestra.run.smithi033.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T01:04:24.521 INFO:teuthology.orchestra.run.smithi033.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T01:04:24.573 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T01:04:24.590 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T01:04:24.592 INFO:teuthology.orchestra.run.smithi033.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T01:04:24.593 INFO:teuthology.orchestra.run.smithi033.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T01:04:24.597 INFO:teuthology.orchestra.run.smithi033.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T01:04:24.598 INFO:teuthology.orchestra.run.smithi033.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T01:04:24.598 INFO:teuthology.orchestra.run.smithi033.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T01:04:24.628 INFO:teuthology.orchestra.run.smithi033.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T01:04:24.632 INFO:teuthology.orchestra.run.smithi033.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T01:04:24.632 INFO:teuthology.orchestra.run.smithi033.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T01:04:24.638 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T01:04:24.667 INFO:teuthology.orchestra.run.smithi033.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T01:04:24.667 INFO:teuthology.orchestra.run.smithi033.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T01:04:24.667 INFO:teuthology.orchestra.run.smithi033.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T01:04:24.677 INFO:teuthology.orchestra.run.smithi033.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T01:04:24.678 INFO:teuthology.orchestra.run.smithi033.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T01:04:24.678 INFO:teuthology.orchestra.run.smithi033.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T01:04:24.679 INFO:teuthology.orchestra.run.smithi033.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T01:04:24.679 INFO:teuthology.orchestra.run.smithi033.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T01:04:24.683 INFO:teuthology.orchestra.run.smithi033.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T01:04:24.703 INFO:teuthology.orchestra.run.smithi033.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T01:04:24.705 INFO:teuthology.orchestra.run.smithi033.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T01:04:24.706 INFO:teuthology.orchestra.run.smithi033.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T01:04:24.709 INFO:teuthology.orchestra.run.smithi033.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T01:04:24.726 INFO:teuthology.orchestra.run.smithi033.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T01:04:24.726 INFO:teuthology.orchestra.run.smithi033.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T01:04:24.727 INFO:teuthology.orchestra.run.smithi033.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T01:04:24.728 INFO:teuthology.orchestra.run.smithi033.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T01:04:24.733 INFO:teuthology.orchestra.run.smithi033.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T01:04:24.734 INFO:teuthology.orchestra.run.smithi033.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T01:04:24.749 INFO:teuthology.orchestra.run.smithi033.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T01:04:24.749 INFO:teuthology.orchestra.run.smithi033.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T01:04:24.749 INFO:teuthology.orchestra.run.smithi033.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T01:04:24.750 INFO:teuthology.orchestra.run.smithi033.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T01:04:24.776 INFO:teuthology.orchestra.run.smithi033.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T01:04:24.777 INFO:teuthology.orchestra.run.smithi033.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T01:04:24.789 INFO:teuthology.orchestra.run.smithi033.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T01:04:24.791 INFO:teuthology.orchestra.run.smithi033.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T01:04:24.791 INFO:teuthology.orchestra.run.smithi033.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T01:04:24.793 INFO:teuthology.orchestra.run.smithi033.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T01:04:24.812 INFO:teuthology.orchestra.run.smithi033.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T01:04:24.818 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T01:04:24.827 INFO:teuthology.orchestra.run.smithi138.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T01:04:24.854 INFO:teuthology.orchestra.run.smithi138.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T01:04:24.863 INFO:teuthology.orchestra.run.smithi138.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T01:04:25.030 INFO:teuthology.orchestra.run.smithi138.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T01:04:25.477 INFO:teuthology.orchestra.run.smithi138.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T01:04:25.612 INFO:teuthology.orchestra.run.smithi138.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T01:04:25.627 INFO:teuthology.orchestra.run.smithi138.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T01:04:25.643 INFO:teuthology.orchestra.run.smithi138.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T01:04:25.651 INFO:teuthology.orchestra.run.smithi138.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T01:04:25.652 INFO:teuthology.orchestra.run.smithi138.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T01:04:25.669 INFO:teuthology.orchestra.run.smithi138.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T01:04:25.706 INFO:teuthology.orchestra.run.smithi138.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T01:04:25.719 INFO:teuthology.orchestra.run.smithi138.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T01:04:25.723 INFO:teuthology.orchestra.run.smithi138.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T01:04:25.723 INFO:teuthology.orchestra.run.smithi138.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T01:04:25.816 INFO:teuthology.orchestra.run.smithi138.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T01:04:25.827 INFO:teuthology.orchestra.run.smithi033.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2445-gd2c45d97-1jammy [3,218 kB] 2024-04-25T01:04:25.885 INFO:teuthology.orchestra.run.smithi138.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T01:04:25.895 INFO:teuthology.orchestra.run.smithi138.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T01:04:25.919 INFO:teuthology.orchestra.run.smithi138.stdout:Get:28 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T01:04:25.920 INFO:teuthology.orchestra.run.smithi138.stdout:Get:29 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T01:04:25.924 INFO:teuthology.orchestra.run.smithi138.stdout:Get:30 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T01:04:25.925 INFO:teuthology.orchestra.run.smithi138.stdout:Get:31 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T01:04:25.958 INFO:teuthology.orchestra.run.smithi138.stdout:Get:32 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T01:04:25.978 INFO:teuthology.orchestra.run.smithi138.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T01:04:25.985 INFO:teuthology.orchestra.run.smithi138.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T01:04:26.060 INFO:teuthology.orchestra.run.smithi138.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T01:04:26.086 INFO:teuthology.orchestra.run.smithi138.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T01:04:26.102 INFO:teuthology.orchestra.run.smithi138.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T01:04:26.119 INFO:teuthology.orchestra.run.smithi138.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T01:04:26.312 INFO:teuthology.orchestra.run.smithi138.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T01:04:26.328 INFO:teuthology.orchestra.run.smithi138.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T01:04:26.335 INFO:teuthology.orchestra.run.smithi138.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T01:04:26.339 INFO:teuthology.orchestra.run.smithi138.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T01:04:26.366 INFO:teuthology.orchestra.run.smithi138.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T01:04:26.369 INFO:teuthology.orchestra.run.smithi138.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T01:04:26.388 INFO:teuthology.orchestra.run.smithi138.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T01:04:26.523 INFO:teuthology.orchestra.run.smithi138.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T01:04:26.606 INFO:teuthology.orchestra.run.smithi138.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T01:04:26.622 INFO:teuthology.orchestra.run.smithi138.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T01:04:26.711 INFO:teuthology.orchestra.run.smithi033.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2445-gd2c45d97-1jammy [711 kB] 2024-04-25T01:04:26.809 INFO:teuthology.orchestra.run.smithi138.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T01:04:26.861 INFO:teuthology.orchestra.run.smithi138.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T01:04:26.875 INFO:teuthology.orchestra.run.smithi138.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T01:04:26.944 INFO:teuthology.orchestra.run.smithi033.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2445-gd2c45d97-1jammy [356 kB] 2024-04-25T01:04:27.037 INFO:teuthology.orchestra.run.smithi138.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T01:04:27.038 INFO:teuthology.orchestra.run.smithi138.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T01:04:27.043 INFO:teuthology.orchestra.run.smithi138.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T01:04:27.048 INFO:teuthology.orchestra.run.smithi138.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T01:04:27.060 INFO:teuthology.orchestra.run.smithi033.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2445-gd2c45d97-1jammy [32.8 kB] 2024-04-25T01:04:27.062 INFO:teuthology.orchestra.run.smithi033.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2445-gd2c45d97-1jammy [180 kB] 2024-04-25T01:04:27.063 INFO:teuthology.orchestra.run.smithi138.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T01:04:27.178 INFO:teuthology.orchestra.run.smithi033.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2445-gd2c45d97-1jammy [65.7 kB] 2024-04-25T01:04:27.184 INFO:teuthology.orchestra.run.smithi033.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2445-gd2c45d97-1jammy [330 kB] 2024-04-25T01:04:27.255 INFO:teuthology.orchestra.run.smithi138.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T01:04:27.323 INFO:teuthology.orchestra.run.smithi033.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2445-gd2c45d97-1jammy [6,775 kB] 2024-04-25T01:04:27.441 INFO:teuthology.orchestra.run.smithi138.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T01:04:27.555 INFO:teuthology.orchestra.run.smithi138.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T01:04:27.567 INFO:teuthology.orchestra.run.smithi138.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T01:04:27.570 INFO:teuthology.orchestra.run.smithi138.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T01:04:27.594 INFO:teuthology.orchestra.run.smithi138.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T01:04:27.610 INFO:teuthology.orchestra.run.smithi138.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T01:04:27.715 INFO:teuthology.orchestra.run.smithi138.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T01:04:27.759 INFO:teuthology.orchestra.run.smithi138.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T01:04:27.830 INFO:teuthology.orchestra.run.smithi138.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T01:04:27.845 INFO:teuthology.orchestra.run.smithi138.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T01:04:27.857 INFO:teuthology.orchestra.run.smithi138.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T01:04:27.870 INFO:teuthology.orchestra.run.smithi138.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T01:04:27.911 INFO:teuthology.orchestra.run.smithi138.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T01:04:28.178 INFO:teuthology.orchestra.run.smithi138.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T01:04:28.179 INFO:teuthology.orchestra.run.smithi138.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T01:04:28.182 INFO:teuthology.orchestra.run.smithi138.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T01:04:28.347 INFO:teuthology.orchestra.run.smithi138.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T01:04:28.691 INFO:teuthology.orchestra.run.smithi138.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T01:04:28.744 INFO:teuthology.orchestra.run.smithi138.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T01:04:28.745 INFO:teuthology.orchestra.run.smithi138.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T01:04:28.762 INFO:teuthology.orchestra.run.smithi138.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T01:04:28.803 INFO:teuthology.orchestra.run.smithi138.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T01:04:28.813 INFO:teuthology.orchestra.run.smithi138.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T01:04:28.815 INFO:teuthology.orchestra.run.smithi138.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T01:04:28.901 INFO:teuthology.orchestra.run.smithi138.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T01:04:29.107 INFO:teuthology.orchestra.run.smithi138.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T01:04:29.117 INFO:teuthology.orchestra.run.smithi138.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T01:04:29.118 INFO:teuthology.orchestra.run.smithi138.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T01:04:29.196 INFO:teuthology.orchestra.run.smithi138.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T01:04:29.196 INFO:teuthology.orchestra.run.smithi138.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T01:04:29.198 INFO:teuthology.orchestra.run.smithi138.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T01:04:29.199 INFO:teuthology.orchestra.run.smithi138.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T01:04:29.200 INFO:teuthology.orchestra.run.smithi138.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T01:04:29.209 INFO:teuthology.orchestra.run.smithi138.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T01:04:29.210 INFO:teuthology.orchestra.run.smithi138.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T01:04:29.373 INFO:teuthology.orchestra.run.smithi138.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T01:04:29.475 INFO:teuthology.orchestra.run.smithi138.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T01:04:29.567 INFO:teuthology.orchestra.run.smithi138.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T01:04:29.811 INFO:teuthology.orchestra.run.smithi138.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T01:04:29.853 INFO:teuthology.orchestra.run.smithi138.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T01:04:29.879 INFO:teuthology.orchestra.run.smithi138.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T01:04:29.887 INFO:teuthology.orchestra.run.smithi138.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T01:04:29.930 INFO:teuthology.orchestra.run.smithi138.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T01:04:29.956 INFO:teuthology.orchestra.run.smithi138.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T01:04:29.958 INFO:teuthology.orchestra.run.smithi138.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T01:04:29.964 INFO:teuthology.orchestra.run.smithi138.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T01:04:30.131 INFO:teuthology.orchestra.run.smithi138.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T01:04:30.195 INFO:teuthology.orchestra.run.smithi033.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2445-gd2c45d97-1jammy [112 kB] 2024-04-25T01:04:30.198 INFO:teuthology.orchestra.run.smithi033.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2445-gd2c45d97-1jammy [466 kB] 2024-04-25T01:04:30.429 INFO:teuthology.orchestra.run.smithi033.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2445-gd2c45d97-1jammy [25.6 MB] 2024-04-25T01:04:30.670 INFO:teuthology.orchestra.run.smithi138.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T01:04:30.676 INFO:teuthology.orchestra.run.smithi138.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T01:04:30.716 INFO:teuthology.orchestra.run.smithi138.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T01:04:30.717 INFO:teuthology.orchestra.run.smithi138.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T01:04:30.723 INFO:teuthology.orchestra.run.smithi138.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T01:04:30.725 INFO:teuthology.orchestra.run.smithi138.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T01:04:30.768 INFO:teuthology.orchestra.run.smithi138.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T01:04:30.776 INFO:teuthology.orchestra.run.smithi138.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T01:04:30.851 INFO:teuthology.orchestra.run.smithi138.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T01:04:30.852 INFO:teuthology.orchestra.run.smithi138.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T01:04:31.016 INFO:teuthology.orchestra.run.smithi138.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T01:04:31.247 INFO:teuthology.orchestra.run.smithi138.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T01:04:31.249 INFO:teuthology.orchestra.run.smithi138.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T01:04:31.254 INFO:teuthology.orchestra.run.smithi138.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T01:04:31.255 INFO:teuthology.orchestra.run.smithi138.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T01:04:31.255 INFO:teuthology.orchestra.run.smithi138.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T01:04:31.256 INFO:teuthology.orchestra.run.smithi138.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T01:04:31.459 INFO:teuthology.orchestra.run.smithi138.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T01:04:31.474 INFO:teuthology.orchestra.run.smithi138.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T01:04:31.633 INFO:teuthology.orchestra.run.smithi138.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T01:04:32.373 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 180 MB in 8s (23.1 MB/s) 2024-04-25T01:04:32.496 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T01:04:32.594 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T01:04:32.600 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T01:04:32.644 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T01:04:33.115 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T01:04:33.140 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T01:04:33.181 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T01:04:33.500 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T01:04:33.528 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T01:04:33.567 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T01:04:33.919 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T01:04:33.946 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T01:04:34.054 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T01:04:34.398 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T01:04:34.425 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T01:04:34.465 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T01:04:34.751 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T01:04:34.778 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T01:04:34.809 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T01:04:35.246 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T01:04:35.273 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T01:04:35.321 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:04:35.724 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T01:04:35.751 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T01:04:35.800 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:04:35.853 INFO:teuthology.orchestra.run.smithi033.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2445-gd2c45d97-1jammy [5,084 kB] 2024-04-25T01:04:36.094 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T01:04:36.120 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T01:04:36.160 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:04:36.447 INFO:teuthology.orchestra.run.smithi033.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2445-gd2c45d97-1jammy [244 kB] 2024-04-25T01:04:36.478 INFO:teuthology.orchestra.run.smithi033.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2445-gd2c45d97-1jammy [124 kB] 2024-04-25T01:04:36.488 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T01:04:36.496 INFO:teuthology.orchestra.run.smithi033.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2445-gd2c45d97-1jammy [1,525 kB] 2024-04-25T01:04:36.515 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T01:04:36.555 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T01:04:36.681 INFO:teuthology.orchestra.run.smithi033.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2445-gd2c45d97-1jammy [6,245 kB] 2024-04-25T01:04:36.875 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librados2. 2024-04-25T01:04:36.901 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:36.941 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:37.369 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librbd1. 2024-04-25T01:04:37.395 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:37.436 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:37.513 INFO:teuthology.orchestra.run.smithi033.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2445-gd2c45d97-1jammy [22.7 MB] 2024-04-25T01:04:37.872 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T01:04:37.898 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:37.938 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:38.215 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rados. 2024-04-25T01:04:38.230 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:38.282 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:38.710 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T01:04:38.736 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:04:38.777 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:39.029 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T01:04:39.044 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:39.079 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:39.365 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T01:04:39.391 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:04:39.432 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:39.643 INFO:teuthology.orchestra.run.smithi033.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2445-gd2c45d97-1jammy [14.0 kB] 2024-04-25T01:04:39.644 INFO:teuthology.orchestra.run.smithi033.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [869 kB] 2024-04-25T01:04:39.660 INFO:teuthology.orchestra.run.smithi033.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2445-gd2c45d97-1jammy [2,437 kB] 2024-04-25T01:04:39.718 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T01:04:39.744 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T01:04:39.785 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T01:04:39.843 INFO:teuthology.orchestra.run.smithi033.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2445-gd2c45d97-1jammy [790 kB] 2024-04-25T01:04:39.982 INFO:teuthology.orchestra.run.smithi033.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2445-gd2c45d97-1jammy [147 kB] 2024-04-25T01:04:39.985 INFO:teuthology.orchestra.run.smithi033.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2445-gd2c45d97-1jammy [2,293 kB] 2024-04-25T01:04:40.038 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T01:04:40.064 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:40.104 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:40.132 INFO:teuthology.orchestra.run.smithi033.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2445-gd2c45d97-1jammy [8,625 kB] 2024-04-25T01:04:40.448 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T01:04:40.475 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T01:04:40.515 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T01:04:40.698 INFO:teuthology.orchestra.run.smithi033.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2445-gd2c45d97-1jammy [14.2 kB] 2024-04-25T01:04:40.698 INFO:teuthology.orchestra.run.smithi033.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2445-gd2c45d97-1jammy [50.7 MB] 2024-04-25T01:04:40.768 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T01:04:40.783 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T01:04:40.817 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T01:04:41.104 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T01:04:41.119 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T01:04:41.170 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T01:04:41.515 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua5.1. 2024-04-25T01:04:41.542 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T01:04:41.582 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T01:04:41.868 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua-any. 2024-04-25T01:04:41.884 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T01:04:41.918 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T01:04:42.162 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package zip. 2024-04-25T01:04:42.189 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T01:04:42.229 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T01:04:42.540 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package unzip. 2024-04-25T01:04:42.556 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T01:04:42.599 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T01:04:42.961 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package luarocks. 2024-04-25T01:04:42.987 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T01:04:43.027 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T01:04:43.377 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package librgw2. 2024-04-25T01:04:43.403 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:43.443 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:43.996 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T01:04:44.022 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:44.062 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:44.399 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T01:04:44.425 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T01:04:44.465 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T01:04:44.776 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T01:04:44.803 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:44.835 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:45.111 INFO:teuthology.orchestra.run.smithi033.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2445-gd2c45d97-1jammy [121 kB] 2024-04-25T01:04:45.114 INFO:teuthology.orchestra.run.smithi033.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2445-gd2c45d97-1jammy [229 kB] 2024-04-25T01:04:45.118 INFO:teuthology.orchestra.run.smithi033.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2445-gd2c45d97-1jammy [34.0 kB] 2024-04-25T01:04:45.119 INFO:teuthology.orchestra.run.smithi033.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2445-gd2c45d97-1jammy [23.4 kB] 2024-04-25T01:04:45.120 INFO:teuthology.orchestra.run.smithi033.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2445-gd2c45d97-1jammy [38.9 kB] 2024-04-25T01:04:45.121 INFO:teuthology.orchestra.run.smithi033.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2445-gd2c45d97-1jammy [13.4 MB] 2024-04-25T01:04:45.230 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-common. 2024-04-25T01:04:45.256 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:45.296 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:46.668 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-base. 2024-04-25T01:04:46.694 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:46.744 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:46.782 INFO:teuthology.orchestra.run.smithi033.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2445-gd2c45d97-1jammy [91.2 kB] 2024-04-25T01:04:47.272 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T01:04:47.291 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T01:04:47.322 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T01:04:47.413 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 180 MB in 23s (7,931 kB/s) 2024-04-25T01:04:47.539 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T01:04:47.617 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T01:04:47.637 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T01:04:47.642 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T01:04:47.643 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T01:04:47.683 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T01:04:47.687 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T01:04:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T01:04:47.990 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T01:04:48.012 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T01:04:48.016 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T01:04:48.052 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T01:04:48.056 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T01:04:48.368 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T01:04:48.380 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T01:04:48.394 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T01:04:48.407 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T01:04:48.434 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T01:04:48.447 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T01:04:48.717 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T01:04:48.743 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T01:04:48.775 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T01:04:48.804 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T01:04:48.831 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T01:04:48.913 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T01:04:49.224 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T01:04:49.240 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T01:04:49.261 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T01:04:49.283 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T01:04:49.287 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T01:04:49.327 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T01:04:49.614 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-portend. 2024-04-25T01:04:49.619 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T01:04:49.640 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T01:04:49.646 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T01:04:49.680 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T01:04:49.686 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T01:04:49.991 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T01:04:50.018 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T01:04:50.058 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T01:04:50.140 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T01:04:50.167 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T01:04:50.214 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:04:50.302 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T01:04:50.329 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T01:04:50.369 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T01:04:50.685 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T01:04:50.706 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T01:04:50.712 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T01:04:50.732 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T01:04:50.751 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:04:50.772 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T01:04:51.084 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T01:04:51.096 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T01:04:51.110 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T01:04:51.123 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T01:04:51.150 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T01:04:51.162 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:04:51.428 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T01:04:51.443 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T01:04:51.478 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T01:04:51.516 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T01:04:51.543 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T01:04:51.582 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-mako. 2024-04-25T01:04:51.783 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T01:04:51.822 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T01:04:51.885 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librados2. 2024-04-25T01:04:51.912 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../010-librados2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:51.952 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:52.176 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T01:04:52.203 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T01:04:52.242 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T01:04:52.405 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librbd1. 2024-04-25T01:04:52.432 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../011-librbd1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:52.472 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:52.545 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T01:04:52.572 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T01:04:52.612 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T01:04:52.889 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-webob. 2024-04-25T01:04:52.916 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T01:04:52.941 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T01:04:52.956 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T01:04:52.968 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:53.008 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:53.252 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rados. 2024-04-25T01:04:53.268 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:53.302 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:53.303 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T01:04:53.329 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T01:04:53.373 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T01:04:53.630 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T01:04:53.657 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:04:53.681 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T01:04:53.696 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:53.707 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T01:04:53.747 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T01:04:53.949 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T01:04:53.976 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:54.015 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:54.062 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-paste. 2024-04-25T01:04:54.086 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T01:04:54.126 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T01:04:54.310 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T01:04:54.337 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:04:54.377 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:54.455 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T01:04:54.470 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T01:04:54.513 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T01:04:54.672 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T01:04:54.688 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T01:04:54.722 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T01:04:54.816 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T01:04:54.843 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T01:04:54.883 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T01:04:54.991 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T01:04:55.007 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:55.049 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:55.145 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T01:04:55.160 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T01:04:55.203 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T01:04:55.444 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T01:04:55.472 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T01:04:55.511 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T01:04:55.789 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T01:04:55.805 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T01:04:55.838 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T01:04:56.041 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T01:04:56.067 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T01:04:56.116 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T01:04:56.132 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T01:04:56.175 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T01:04:56.357 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T01:04:56.503 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package lua5.1. 2024-04-25T01:04:56.530 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T01:04:56.570 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T01:04:56.705 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T01:04:56.731 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T01:04:56.771 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T01:04:56.881 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package lua-any. 2024-04-25T01:04:56.897 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T01:04:56.931 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T01:04:57.058 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-idna. 2024-04-25T01:04:57.085 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T01:04:57.116 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T01:04:57.150 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package zip. 2024-04-25T01:04:57.167 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T01:04:57.209 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T01:04:57.412 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T01:04:57.438 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T01:04:57.478 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T01:04:57.782 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-requests. 2024-04-25T01:04:57.808 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T01:04:57.848 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T01:04:58.177 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T01:04:58.203 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T01:04:58.221 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package unzip. 2024-04-25T01:04:58.243 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T01:04:58.248 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T01:04:58.288 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T01:04:58.530 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T01:04:58.557 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:04:58.597 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:58.642 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package luarocks. 2024-04-25T01:04:58.668 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T01:04:58.700 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T01:04:58.986 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T01:04:59.013 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:59.058 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package librgw2. 2024-04-25T01:04:59.085 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../027-librgw2_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:59.086 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:59.124 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:59.548 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T01:04:59.574 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T01:04:59.614 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T01:04:59.685 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T01:04:59.712 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:04:59.752 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:04:59.901 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T01:04:59.927 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T01:04:59.959 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T01:05:00.088 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T01:05:00.115 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T01:05:00.155 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T01:05:00.280 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T01:05:00.307 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:00.346 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:00.408 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T01:05:00.423 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:00.457 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:00.733 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T01:05:00.760 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:00.802 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:00.886 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-common. 2024-04-25T01:05:00.913 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:00.952 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:01.329 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T01:05:01.355 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:01.387 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:02.308 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-base. 2024-04-25T01:05:02.335 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:02.384 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:02.516 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph. 2024-04-25T01:05:02.542 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:02.582 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:02.927 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T01:05:02.953 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:02.993 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:03.006 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T01:05:03.033 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T01:05:03.081 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T01:05:03.322 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T01:05:03.349 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:03.380 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:03.417 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T01:05:03.444 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T01:05:03.484 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T01:05:03.792 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephadm. 2024-04-25T01:05:03.812 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T01:05:03.818 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:03.839 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T01:05:03.858 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:03.878 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T01:05:04.172 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T01:05:04.190 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T01:05:04.197 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T01:05:04.219 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T01:05:04.237 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T01:05:04.265 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T01:05:04.515 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T01:05:04.530 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T01:05:04.573 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T01:05:04.609 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T01:05:04.636 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T01:05:04.684 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T01:05:05.028 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T01:05:05.055 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T01:05:05.095 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T01:05:05.398 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-portend. 2024-04-25T01:05:05.425 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T01:05:05.464 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T01:05:05.596 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-babel. 2024-04-25T01:05:05.623 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T01:05:05.662 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T01:05:05.767 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T01:05:05.794 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T01:05:05.833 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T01:05:05.965 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T01:05:05.992 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T01:05:06.032 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T01:05:06.187 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T01:05:06.214 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T01:05:06.261 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T01:05:06.385 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T01:05:06.412 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:06.452 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:06.657 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T01:05:06.684 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T01:05:06.723 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T01:05:06.797 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T01:05:06.824 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T01:05:06.864 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T01:05:07.126 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T01:05:07.154 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T01:05:07.193 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T01:05:07.276 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-routes. 2024-04-25T01:05:07.302 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T01:05:07.342 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T01:05:07.563 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T01:05:07.590 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T01:05:07.637 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T01:05:07.671 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T01:05:07.698 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:07.737 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:07.924 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-mako. 2024-04-25T01:05:07.940 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T01:05:07.982 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T01:05:08.310 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T01:05:08.337 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T01:05:08.376 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T01:05:08.663 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T01:05:08.690 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T01:05:08.729 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T01:05:08.977 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T01:05:08.993 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T01:05:09.035 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T01:05:09.040 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webob. 2024-04-25T01:05:09.068 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T01:05:09.107 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T01:05:09.427 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T01:05:09.454 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T01:05:09.497 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T01:05:09.514 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T01:05:09.542 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T01:05:09.572 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T01:05:09.754 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T01:05:09.770 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T01:05:09.812 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T01:05:10.110 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T01:05:10.124 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-paste. 2024-04-25T01:05:10.137 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T01:05:10.151 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T01:05:10.190 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T01:05:10.262 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T01:05:10.535 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T01:05:10.563 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T01:05:10.566 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T01:05:10.593 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T01:05:10.601 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T01:05:10.632 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T01:05:10.838 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T01:05:10.865 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T01:05:10.896 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T01:05:11.199 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T01:05:11.215 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T01:05:11.257 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T01:05:11.285 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T01:05:11.312 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:11.351 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:11.568 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T01:05:11.596 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T01:05:11.635 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T01:05:11.998 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T01:05:12.025 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T01:05:12.073 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T01:05:12.322 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T01:05:12.349 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T01:05:12.388 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T01:05:12.401 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-idna. 2024-04-25T01:05:12.428 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T01:05:12.467 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T01:05:12.692 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T01:05:12.719 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T01:05:12.737 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T01:05:12.758 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T01:05:12.764 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T01:05:12.795 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T01:05:13.054 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T01:05:13.081 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T01:05:13.112 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T01:05:13.148 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-requests. 2024-04-25T01:05:13.176 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T01:05:13.206 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T01:05:13.432 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T01:05:13.459 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T01:05:13.493 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T01:05:13.499 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T01:05:13.520 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T01:05:13.559 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T01:05:13.803 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T01:05:13.831 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T01:05:13.845 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T01:05:13.862 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:13.870 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T01:05:13.895 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:14.190 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T01:05:14.217 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T01:05:14.257 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T01:05:14.267 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T01:05:14.294 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:14.341 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:14.535 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T01:05:14.562 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T01:05:14.602 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T01:05:14.653 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T01:05:14.680 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T01:05:14.719 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T01:05:14.923 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T01:05:14.950 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T01:05:14.980 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T01:05:14.989 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T01:05:14.996 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T01:05:15.030 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T01:05:15.375 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T01:05:15.402 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:15.441 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:15.662 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T01:05:15.679 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:15.846 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:16.011 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T01:05:16.038 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:16.086 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:16.157 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T01:05:16.184 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T01:05:16.215 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T01:05:16.552 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T01:05:16.579 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T01:05:16.618 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T01:05:16.689 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T01:05:16.716 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:16.755 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:16.871 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package jq. 2024-04-25T01:05:16.887 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T01:05:16.930 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T01:05:17.183 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package socat. 2024-04-25T01:05:17.199 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T01:05:17.241 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T01:05:17.545 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T01:05:17.561 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T01:05:17.603 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T01:05:17.859 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph. 2024-04-25T01:05:17.875 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../068-ceph_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:17.925 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:17.941 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-test. 2024-04-25T01:05:17.958 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:18.000 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:18.320 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T01:05:18.347 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:18.386 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:18.773 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T01:05:18.800 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:18.848 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:19.234 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephadm. 2024-04-25T01:05:19.262 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../071-cephadm_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:19.300 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:19.578 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T01:05:19.607 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T01:05:19.644 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T01:05:19.964 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T01:05:19.993 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T01:05:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T01:05:20.722 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T01:05:20.749 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:20.788 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:21.018 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-babel. 2024-04-25T01:05:21.046 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T01:05:21.076 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T01:05:21.151 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T01:05:21.179 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:21.218 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:21.538 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T01:05:21.565 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T01:05:21.605 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T01:05:21.705 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T01:05:21.732 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T01:05:21.771 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T01:05:21.866 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T01:05:21.882 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:21.924 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:22.158 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T01:05:22.177 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T01:05:22.185 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:22.193 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:22.224 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:22.235 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:22.472 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T01:05:22.488 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:22.522 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:22.561 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T01:05:22.588 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T01:05:22.627 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T01:05:22.842 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T01:05:22.869 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T01:05:22.908 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T01:05:22.930 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-routes. 2024-04-25T01:05:22.957 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T01:05:22.996 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T01:05:23.239 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T01:05:23.266 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T01:05:23.305 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T01:05:23.350 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T01:05:23.377 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:23.416 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:23.668 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T01:05:23.695 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T01:05:23.734 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T01:05:24.182 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package pkg-config. 2024-04-25T01:05:24.210 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T01:05:24.249 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T01:05:24.511 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T01:05:24.538 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T01:05:24.577 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T01:05:24.696 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T01:05:24.724 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T01:05:24.762 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T01:05:25.014 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T01:05:25.042 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T01:05:25.081 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T01:05:25.291 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T01:05:25.319 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T01:05:25.357 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T01:05:25.359 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T01:05:25.386 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T01:05:25.417 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T01:05:25.711 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T01:05:25.721 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T01:05:25.739 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T01:05:25.749 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T01:05:25.788 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T01:05:25.888 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T01:05:26.099 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T01:05:26.127 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T01:05:26.166 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T01:05:26.199 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T01:05:26.227 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T01:05:26.265 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T01:05:26.494 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-py. 2024-04-25T01:05:26.522 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T01:05:26.569 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T01:05:26.898 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T01:05:26.926 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T01:05:26.965 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T01:05:27.033 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T01:05:27.061 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:27.099 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:27.419 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T01:05:27.447 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T01:05:27.477 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T01:05:27.814 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-toml. 2024-04-25T01:05:27.842 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T01:05:27.881 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T01:05:28.102 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T01:05:28.130 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T01:05:28.168 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T01:05:28.184 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T01:05:28.211 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T01:05:28.250 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T01:05:28.446 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T01:05:28.474 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T01:05:28.512 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T01:05:28.596 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T01:05:28.624 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T01:05:28.662 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T01:05:28.825 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T01:05:28.852 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T01:05:28.882 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T01:05:28.915 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T01:05:28.932 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T01:05:28.974 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T01:05:29.152 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T01:05:29.180 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T01:05:29.210 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T01:05:29.531 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T01:05:29.559 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T01:05:29.597 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T01:05:29.712 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package radosgw. 2024-04-25T01:05:29.740 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:29.770 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:29.942 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T01:05:29.970 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T01:05:30.008 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T01:05:30.320 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T01:05:30.348 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T01:05:30.386 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T01:05:30.600 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T01:05:30.616 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:30.658 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:30.690 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T01:05:30.718 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T01:05:30.748 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T01:05:31.164 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T01:05:31.407 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T01:05:31.436 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T01:05:31.453 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:31.533 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T01:05:31.772 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T01:05:31.789 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T01:05:31.800 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T01:05:31.830 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T01:05:32.123 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T01:05:32.159 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T01:05:32.187 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T01:05:32.225 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T01:05:32.494 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package jq. 2024-04-25T01:05:32.511 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T01:05:32.543 INFO:teuthology.orchestra.run.smithi138.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T01:05:32.552 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T01:05:32.635 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:32.805 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package socat. 2024-04-25T01:05:32.822 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T01:05:32.855 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T01:05:33.127 INFO:teuthology.orchestra.run.smithi138.stdout:Adding system user cephadm....done 2024-04-25T01:05:33.142 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T01:05:33.158 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T01:05:33.200 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T01:05:33.242 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T01:05:33.510 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T01:05:33.579 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-test. 2024-04-25T01:05:33.596 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:33.637 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:33.762 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T01:05:33.863 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T01:05:34.106 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T01:05:34.378 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T01:05:34.501 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T01:05:34.803 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T01:05:35.199 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T01:05:35.456 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T01:05:35.707 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T01:05:35.941 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T01:05:35.958 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:35.975 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T01:05:35.991 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:36.211 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T01:05:36.320 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T01:05:36.348 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:36.378 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:36.471 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T01:05:36.681 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T01:05:36.709 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T01:05:36.730 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T01:05:36.756 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T01:05:37.073 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:37.300 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T01:05:37.316 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T01:05:37.374 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:37.459 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T01:05:37.585 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T01:05:37.644 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T01:05:37.660 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2445-gd2c45d97-1jammy_all.deb ... 2024-04-25T01:05:37.702 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:37.845 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T01:05:37.946 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T01:05:37.962 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:37.971 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T01:05:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:38.222 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T01:05:38.274 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T01:05:38.302 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T01:05:38.340 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T01:05:38.610 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T01:05:38.663 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T01:05:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T01:05:38.698 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T01:05:38.709 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T01:05:38.736 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T01:05:38.819 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T01:05:39.115 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T01:05:39.143 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T01:05:39.181 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T01:05:39.195 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T01:05:39.322 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T01:05:39.590 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T01:05:39.595 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package pkg-config. 2024-04-25T01:05:39.623 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T01:05:39.653 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T01:05:39.858 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T01:05:39.872 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T01:05:39.889 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T01:05:39.930 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T01:05:40.135 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T01:05:40.261 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T01:05:40.300 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T01:05:40.317 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T01:05:40.350 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T01:05:40.644 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T01:05:40.672 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T01:05:40.711 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T01:05:40.946 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T01:05:41.031 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T01:05:41.059 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T01:05:41.097 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T01:05:41.214 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T01:05:41.366 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T01:05:41.434 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T01:05:41.462 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T01:05:41.492 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T01:05:41.500 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T01:05:41.785 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T01:05:41.836 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-py. 2024-04-25T01:05:41.864 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T01:05:41.903 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T01:05:42.161 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T01:05:42.373 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T01:05:42.396 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T01:05:42.401 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T01:05:42.441 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T01:05:42.766 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T01:05:42.904 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T01:05:42.932 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T01:05:42.970 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T01:05:43.050 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T01:05:43.281 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-toml. 2024-04-25T01:05:43.309 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T01:05:43.347 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T01:05:43.359 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T01:05:43.659 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T01:05:43.687 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T01:05:43.694 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T01:05:43.725 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T01:05:43.937 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T01:05:44.038 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:44.078 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T01:05:44.107 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T01:05:44.145 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T01:05:44.332 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T01:05:44.414 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T01:05:44.442 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T01:05:44.480 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T01:05:44.650 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T01:05:45.268 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package radosgw. 2024-04-25T01:05:45.296 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../122-radosgw_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:45.334 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:45.865 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T01:05:46.109 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:05:46.179 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T01:05:46.207 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2445-gd2c45d97-1jammy_amd64.deb ... 2024-04-25T01:05:46.243 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T01:05:46.247 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:46.503 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T01:05:46.559 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T01:05:46.629 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T01:05:46.764 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T01:05:46.876 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T01:05:47.002 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T01:05:47.049 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T01:05:47.269 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T01:05:47.308 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T01:05:47.559 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:05:47.606 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T01:05:47.686 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T01:05:47.979 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T01:05:48.055 INFO:teuthology.orchestra.run.smithi033.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T01:05:48.153 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:48.247 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T01:05:48.515 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T01:05:48.695 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user cephadm....done 2024-04-25T01:05:48.774 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T01:05:48.968 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T01:05:49.026 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T01:05:49.285 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T01:05:49.403 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T01:05:49.454 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T01:05:49.713 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T01:05:49.721 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T01:05:49.823 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T01:05:49.847 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T01:05:50.065 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T01:05:50.141 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T01:05:50.350 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T01:05:50.434 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T01:05:50.476 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T01:05:50.685 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T01:05:50.794 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T01:05:50.811 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T01:05:50.912 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T01:05:50.984 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T01:05:51.163 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T01:05:51.231 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T01:05:51.357 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T01:05:51.422 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T01:05:51.674 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T01:05:51.742 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T01:05:51.950 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T01:05:51.993 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T01:05:52.185 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T01:05:52.362 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:05:52.480 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T01:05:52.543 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T01:05:52.824 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T01:05:53.116 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T01:05:53.162 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T01:05:53.393 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T01:05:53.429 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-argparse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:53.596 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T01:05:53.689 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T01:05:53.790 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T01:05:53.856 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T01:05:53.899 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T01:05:54.125 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T01:05:54.159 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T01:05:54.284 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T01:05:54.401 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T01:05:54.544 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T01:05:54.795 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T01:05:54.941 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T01:05:54.974 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T01:05:55.038 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T01:05:55.063 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T01:05:55.139 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T01:05:55.416 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T01:05:55.491 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T01:05:55.542 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:55.617 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T01:05:55.669 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:55.778 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T01:05:55.885 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T01:05:56.082 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:56.160 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T01:05:56.208 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:56.334 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T01:05:56.470 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T01:05:56.613 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T01:05:56.627 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:56.762 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:56.897 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:57.111 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T01:05:57.111 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T01:05:57.289 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T01:05:57.532 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T01:05:57.599 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:57.658 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T01:05:57.725 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:57.784 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T01:05:57.851 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:57.960 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:58.086 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T01:05:58.110 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T01:05:58.503 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T01:05:58.754 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T01:05:59.114 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T01:05:59.283 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:59.399 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T01:05:59.409 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:59.535 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:59.653 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:59.717 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T01:05:59.754 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:05:59.855 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:00.068 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T01:06:00.336 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T01:06:00.441 INFO:teuthology.orchestra.run.smithi138.stdout:Adding group ceph....done 2024-04-25T01:06:00.478 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:00.788 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T01:06:00.792 INFO:teuthology.orchestra.run.smithi138.stdout:Adding system user ceph....done 2024-04-25T01:06:00.921 INFO:teuthology.orchestra.run.smithi138.stdout:Setting system user ceph properties....done 2024-04-25T01:06:00.939 INFO:teuthology.orchestra.run.smithi138.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T01:06:01.077 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T01:06:01.089 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T01:06:01.420 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T01:06:02.058 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:02.280 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:06:02.280 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:06:02.303 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T01:06:02.546 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:06:02.655 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T01:06:02.735 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:02.861 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:02.906 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T01:06:03.032 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T01:06:03.158 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T01:06:03.304 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:06:03.304 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:06:03.460 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T01:06:03.719 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T01:06:03.738 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:03.987 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:06:04.078 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T01:06:04.104 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T01:06:04.389 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T01:06:04.540 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:04.634 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T01:06:04.731 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T01:06:04.731 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T01:06:04.891 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T01:06:05.151 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T01:06:05.200 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:05.410 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T01:06:05.416 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T01:06:05.416 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T01:06:05.694 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T01:06:05.820 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T01:06:05.935 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:06.088 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T01:06:06.189 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T01:06:06.214 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T01:06:06.214 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T01:06:06.474 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T01:06:06.721 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:06.783 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T01:06:06.855 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:07.032 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:07.051 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T01:06:07.177 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T01:06:07.237 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T01:06:07.237 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T01:06:07.302 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T01:06:07.392 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T01:06:07.638 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T01:06:07.709 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:07.738 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T01:06:07.860 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:07.969 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:08.096 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:08.106 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T01:06:08.432 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T01:06:08.533 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T01:06:08.558 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T01:06:08.678 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T01:06:08.684 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T01:06:08.943 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T01:06:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T01:06:09.395 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T01:06:09.704 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T01:06:09.772 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T01:06:09.989 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T01:06:10.175 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T01:06:10.409 INFO:teuthology.orchestra.run.smithi138.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T01:06:10.412 DEBUG:teuthology.orchestra.run.smithi138:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T01:06:10.443 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T01:06:10.468 INFO:teuthology.orchestra.run.smithi138.stdout:Reading package lists... 2024-04-25T01:06:10.670 INFO:teuthology.orchestra.run.smithi138.stdout:Building dependency tree... 2024-04-25T01:06:10.671 INFO:teuthology.orchestra.run.smithi138.stdout:Reading state information... 2024-04-25T01:06:10.719 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T01:06:10.862 INFO:teuthology.orchestra.run.smithi138.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T01:06:10.862 INFO:teuthology.orchestra.run.smithi138.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T01:06:10.862 INFO:teuthology.orchestra.run.smithi138.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T01:06:10.862 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T01:06:10.862 INFO:teuthology.orchestra.run.smithi138.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T01:06:10.862 INFO:teuthology.orchestra.run.smithi138.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T01:06:10.862 INFO:teuthology.orchestra.run.smithi138.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T01:06:10.863 INFO:teuthology.orchestra.run.smithi138.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T01:06:10.863 INFO:teuthology.orchestra.run.smithi138.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T01:06:10.863 INFO:teuthology.orchestra.run.smithi138.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T01:06:10.864 INFO:teuthology.orchestra.run.smithi138.stdout:The following additional packages will be installed: 2024-04-25T01:06:10.864 INFO:teuthology.orchestra.run.smithi138.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T01:06:10.864 INFO:teuthology.orchestra.run.smithi138.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T01:06:10.864 INFO:teuthology.orchestra.run.smithi138.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T01:06:10.864 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T01:06:10.864 INFO:teuthology.orchestra.run.smithi138.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T01:06:10.865 INFO:teuthology.orchestra.run.smithi138.stdout:Suggested packages: 2024-04-25T01:06:10.865 INFO:teuthology.orchestra.run.smithi138.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T01:06:10.865 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T01:06:10.865 INFO:teuthology.orchestra.run.smithi138.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T01:06:10.905 INFO:teuthology.orchestra.run.smithi138.stdout:The following NEW packages will be installed: 2024-04-25T01:06:10.905 INFO:teuthology.orchestra.run.smithi138.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T01:06:10.906 INFO:teuthology.orchestra.run.smithi138.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T01:06:10.906 INFO:teuthology.orchestra.run.smithi138.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T01:06:10.906 INFO:teuthology.orchestra.run.smithi138.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T01:06:10.906 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T01:06:10.907 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T01:06:10.907 INFO:teuthology.orchestra.run.smithi138.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T01:06:10.907 INFO:teuthology.orchestra.run.smithi138.stdout:The following packages will be upgraded: 2024-04-25T01:06:10.908 INFO:teuthology.orchestra.run.smithi138.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T01:06:10.987 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T01:06:11.089 INFO:teuthology.orchestra.run.smithi138.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T01:06:11.089 INFO:teuthology.orchestra.run.smithi138.stdout:Need to get 55.7 MB of archives. 2024-04-25T01:06:11.089 INFO:teuthology.orchestra.run.smithi138.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T01:06:11.089 INFO:teuthology.orchestra.run.smithi138.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T01:06:11.356 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T01:06:11.658 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T01:06:11.724 INFO:teuthology.orchestra.run.smithi138.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T01:06:11.740 INFO:teuthology.orchestra.run.smithi138.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T01:06:11.866 INFO:teuthology.orchestra.run.smithi138.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T01:06:11.867 INFO:teuthology.orchestra.run.smithi138.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T01:06:11.868 INFO:teuthology.orchestra.run.smithi138.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T01:06:11.897 INFO:teuthology.orchestra.run.smithi138.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T01:06:11.898 INFO:teuthology.orchestra.run.smithi138.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T01:06:11.902 INFO:teuthology.orchestra.run.smithi138.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T01:06:11.905 INFO:teuthology.orchestra.run.smithi138.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T01:06:11.906 INFO:teuthology.orchestra.run.smithi138.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T01:06:12.001 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T01:06:12.072 INFO:teuthology.orchestra.run.smithi138.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T01:06:12.136 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librados2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:12.262 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:12.366 INFO:teuthology.orchestra.run.smithi138.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T01:06:12.375 INFO:teuthology.orchestra.run.smithi138.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T01:06:12.379 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T01:06:12.380 INFO:teuthology.orchestra.run.smithi138.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T01:06:12.465 INFO:teuthology.orchestra.run.smithi138.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T01:06:12.674 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:12.800 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libradosstriper1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:12.968 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T01:06:13.261 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librbd1 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:13.379 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-modules-core (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:13.479 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:13.682 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T01:06:13.682 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T01:06:14.172 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcephfs-dev (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:14.282 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rados (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:14.408 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up librgw2 (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:14.534 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephfs-top (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:14.660 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T01:06:15.955 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rbd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:16.055 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up rbd-fuse (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:16.157 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-rgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:16.283 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up python3-cephfs (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:16.409 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephfs-shell (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:16.535 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-common (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:17.069 INFO:teuthology.orchestra.run.smithi033.stdout:Adding group ceph....done 2024-04-25T01:06:17.454 INFO:teuthology.orchestra.run.smithi033.stdout:Adding system user ceph....done 2024-04-25T01:06:17.624 INFO:teuthology.orchestra.run.smithi033.stdout:Setting system user ceph properties....done 2024-04-25T01:06:17.642 INFO:teuthology.orchestra.run.smithi033.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T01:06:17.784 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T01:06:18.089 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T01:06:18.093 INFO:teuthology.orchestra.run.smithi138.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T01:06:18.098 INFO:teuthology.orchestra.run.smithi138.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T01:06:18.098 INFO:teuthology.orchestra.run.smithi138.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T01:06:18.105 INFO:teuthology.orchestra.run.smithi138.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T01:06:18.155 INFO:teuthology.orchestra.run.smithi138.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T01:06:18.165 INFO:teuthology.orchestra.run.smithi138.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T01:06:18.333 INFO:teuthology.orchestra.run.smithi138.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T01:06:18.561 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up cephfs-mirror (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:18.755 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:06:18.755 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T01:06:18.934 INFO:teuthology.orchestra.run.smithi138.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T01:06:18.935 INFO:teuthology.orchestra.run.smithi138.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T01:06:19.117 INFO:teuthology.orchestra.run.smithi138.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T01:06:19.118 INFO:teuthology.orchestra.run.smithi138.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T01:06:19.154 INFO:teuthology.orchestra.run.smithi138.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T01:06:19.204 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-test (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:19.217 INFO:teuthology.orchestra.run.smithi138.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T01:06:19.224 INFO:teuthology.orchestra.run.smithi138.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T01:06:19.225 INFO:teuthology.orchestra.run.smithi138.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T01:06:19.226 INFO:teuthology.orchestra.run.smithi138.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T01:06:19.227 INFO:teuthology.orchestra.run.smithi138.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T01:06:19.305 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up radosgw (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:19.432 INFO:teuthology.orchestra.run.smithi138.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T01:06:19.762 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:06:19.762 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:06:19.957 INFO:teuthology.orchestra.run.smithi138.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T01:06:20.215 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-base (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:20.582 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T01:06:21.025 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mds (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:21.308 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T01:06:21.308 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T01:06:21.785 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:22.012 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T01:06:22.012 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T01:06:22.528 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-osd (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:22.812 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T01:06:22.812 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T01:06:23.313 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:23.463 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:23.623 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mon (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:23.896 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T01:06:23.896 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T01:06:24.392 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-cephadm (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:24.552 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:24.703 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-mgr-dashboard (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:24.854 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ceph-volume (19.0.0-2445-gd2c45d97-1jammy) ... 2024-04-25T01:06:25.200 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T01:06:25.393 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T01:06:25.662 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T01:06:26.794 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T01:06:27.446 INFO:teuthology.orchestra.run.smithi033.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T01:06:27.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T01:06:27.511 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T01:06:27.717 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-04-25T01:06:27.718 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-04-25T01:06:27.910 INFO:teuthology.orchestra.run.smithi033.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T01:06:27.910 INFO:teuthology.orchestra.run.smithi033.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T01:06:27.910 INFO:teuthology.orchestra.run.smithi033.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T01:06:27.910 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T01:06:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T01:06:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T01:06:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T01:06:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T01:06:27.911 INFO:teuthology.orchestra.run.smithi033.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T01:06:27.911 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T01:06:27.912 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-04-25T01:06:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T01:06:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T01:06:27.912 INFO:teuthology.orchestra.run.smithi033.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T01:06:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T01:06:27.913 INFO:teuthology.orchestra.run.smithi033.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T01:06:27.914 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-04-25T01:06:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T01:06:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T01:06:27.914 INFO:teuthology.orchestra.run.smithi033.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T01:06:27.954 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-04-25T01:06:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T01:06:27.955 INFO:teuthology.orchestra.run.smithi033.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T01:06:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T01:06:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T01:06:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T01:06:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T01:06:27.956 INFO:teuthology.orchestra.run.smithi033.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T01:06:27.957 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2024-04-25T01:06:27.957 INFO:teuthology.orchestra.run.smithi033.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T01:06:28.016 INFO:teuthology.orchestra.run.smithi033.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T01:06:28.016 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 55.7 MB of archives. 2024-04-25T01:06:28.016 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T01:06:28.016 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T01:06:28.208 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T01:06:28.217 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T01:06:28.263 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T01:06:28.264 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T01:06:28.265 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T01:06:28.275 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T01:06:28.276 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T01:06:28.278 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T01:06:28.280 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T01:06:28.280 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T01:06:28.281 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T01:06:28.285 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T01:06:28.304 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T01:06:28.304 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T01:06:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T01:06:28.854 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T01:06:28.866 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T01:06:28.866 INFO:teuthology.orchestra.run.smithi033.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T01:06:28.872 INFO:teuthology.orchestra.run.smithi033.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T01:06:28.873 INFO:teuthology.orchestra.run.smithi033.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T01:06:28.883 INFO:teuthology.orchestra.run.smithi033.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T01:06:28.887 INFO:teuthology.orchestra.run.smithi033.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T01:06:28.904 INFO:teuthology.orchestra.run.smithi033.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T01:06:28.904 INFO:teuthology.orchestra.run.smithi033.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T01:06:28.942 INFO:teuthology.orchestra.run.smithi033.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T01:06:28.943 INFO:teuthology.orchestra.run.smithi033.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T01:06:28.956 INFO:teuthology.orchestra.run.smithi033.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T01:06:28.978 INFO:teuthology.orchestra.run.smithi033.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T01:06:28.981 INFO:teuthology.orchestra.run.smithi033.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T01:06:28.981 INFO:teuthology.orchestra.run.smithi033.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T01:06:28.982 INFO:teuthology.orchestra.run.smithi033.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T01:06:28.982 INFO:teuthology.orchestra.run.smithi033.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T01:06:29.023 INFO:teuthology.orchestra.run.smithi033.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T01:06:29.024 INFO:teuthology.orchestra.run.smithi033.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T01:06:29.349 INFO:teuthology.orchestra.run.smithi033.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T01:06:29.349 INFO:teuthology.orchestra.run.smithi033.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T01:06:29.349 INFO:teuthology.orchestra.run.smithi033.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T01:06:29.350 INFO:teuthology.orchestra.run.smithi033.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T01:06:29.350 INFO:teuthology.orchestra.run.smithi033.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T01:06:29.350 INFO:teuthology.orchestra.run.smithi033.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T01:06:29.365 INFO:teuthology.orchestra.run.smithi033.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T01:06:29.365 INFO:teuthology.orchestra.run.smithi033.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T01:06:29.786 INFO:teuthology.orchestra.run.smithi033.stdout:Preconfiguring packages ... 2024-04-25T01:06:29.942 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 55.7 MB in 1s (39.8 MB/s) 2024-04-25T01:06:30.208 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T01:06:30.214 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T01:06:30.359 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T01:06:30.963 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T01:06:31.921 INFO:teuthology.orchestra.run.smithi033.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T01:06:32.354 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package flex. 2024-04-25T01:06:32.441 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T01:06:32.447 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T01:06:32.486 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T01:06:32.995 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T01:06:33.167 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T01:06:33.595 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T01:06:34.240 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T01:06:34.246 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T01:06:34.376 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T01:06:34.762 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T01:06:34.893 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T01:06:35.270 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T01:06:35.686 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package bison. 2024-04-25T01:06:35.795 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T01:06:35.801 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T01:06:35.855 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T01:06:36.323 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T01:06:36.351 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T01:06:36.389 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T01:06:36.734 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package dump. 2024-04-25T01:06:36.785 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T01:06:36.800 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T01:06:37.063 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package indent. 2024-04-25T01:06:37.079 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:37.121 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T01:06:37.374 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T01:06:37.391 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T01:06:37.432 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T01:06:37.753 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T01:06:37.781 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T01:06:37.819 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T01:06:38.081 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T01:06:38.098 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T01:06:38.139 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T01:06:38.526 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T01:06:38.555 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T01:06:38.593 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T01:06:38.854 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T01:06:38.882 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T01:06:38.920 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T01:06:39.624 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T01:06:39.653 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T01:06:39.690 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T01:06:40.002 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T01:06:40.031 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T01:06:40.069 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T01:06:41.839 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T01:06:41.867 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T01:06:41.913 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:06:42.182 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T01:06:42.199 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T01:06:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:06:42.569 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T01:06:42.598 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T01:06:42.661 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:06:42.885 INFO:teuthology.orchestra.run.smithi138.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T01:06:42.963 INFO:teuthology.orchestra.run.smithi138.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T01:06:42.963 INFO:teuthology.orchestra.run.smithi138.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T01:06:42.964 INFO:teuthology.orchestra.run.smithi138.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T01:06:42.993 INFO:teuthology.orchestra.run.smithi138.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T01:06:42.994 INFO:teuthology.orchestra.run.smithi138.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T01:06:43.080 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T01:06:43.109 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T01:06:43.155 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T01:06:43.509 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T01:06:43.537 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T01:06:43.575 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T01:06:43.928 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T01:06:43.957 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:43.995 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:06:44.398 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T01:06:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:44.473 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:06:44.786 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T01:06:44.802 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:44.844 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:06:45.088 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T01:06:45.105 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T01:06:45.146 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T01:06:45.251 INFO:teuthology.orchestra.run.smithi138.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T01:06:45.275 INFO:teuthology.orchestra.run.smithi138.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T01:06:45.667 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T01:06:45.695 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T01:06:45.713 INFO:teuthology.orchestra.run.smithi138.stdout:Preconfiguring packages ... 2024-04-25T01:06:45.750 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T01:06:45.858 INFO:teuthology.orchestra.run.smithi138.stdout:Fetched 55.7 MB in 34s (1,620 kB/s) 2024-04-25T01:06:46.119 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T01:06:46.125 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T01:06:46.153 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package modemmanager. 2024-04-25T01:06:46.182 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T01:06:46.220 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:06:46.264 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T01:06:46.902 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package network-manager. 2024-04-25T01:06:46.906 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T01:06:46.931 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T01:06:46.977 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T01:06:47.535 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ppp. 2024-04-25T01:06:47.564 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T01:06:47.850 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T01:06:47.883 INFO:teuthology.orchestra.run.smithi138.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T01:06:48.140 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package flex. 2024-04-25T01:06:48.181 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T01:06:48.210 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T01:06:48.212 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173984 files and directories currently installed.) 2024-04-25T01:06:48.217 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T01:06:48.248 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T01:06:48.263 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T01:06:48.493 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T01:06:48.510 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T01:06:48.543 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T01:06:48.747 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T01:06:48.796 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T01:06:48.824 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T01:06:48.862 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T01:06:48.887 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T01:06:49.166 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T01:06:49.194 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:49.233 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:06:49.278 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T01:06:49.795 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T01:06:49.800 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T01:06:49.931 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T01:06:50.114 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T01:06:50.145 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T01:06:50.191 INFO:teuthology.orchestra.run.smithi033.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T01:06:50.220 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking postgresql-common (238) ... 2024-04-25T01:06:50.364 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T01:06:50.504 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T01:06:50.688 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T01:06:50.717 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:50.758 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:06:50.831 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T01:06:51.205 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package bison. 2024-04-25T01:06:51.309 INFO:teuthology.orchestra.run.smithi138.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174057 files and directories currently installed.) 2024-04-25T01:06:51.315 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T01:06:51.365 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T01:06:51.749 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T01:06:51.777 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T01:06:51.816 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T01:06:52.025 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package postgresql. 2024-04-25T01:06:52.042 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T01:06:52.083 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking postgresql (14+238) ... 2024-04-25T01:06:52.152 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package dump. 2024-04-25T01:06:52.181 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T01:06:52.219 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T01:06:52.336 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T01:06:52.364 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T01:06:52.402 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T01:06:52.581 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package indent. 2024-04-25T01:06:52.608 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:52.613 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T01:06:52.630 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T01:06:52.647 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T01:06:52.663 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T01:06:52.883 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T01:06:52.899 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T01:06:52.933 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T01:06:52.966 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T01:06:52.994 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T01:06:53.032 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T01:06:53.262 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T01:06:53.290 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T01:06:53.318 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package tcl. 2024-04-25T01:06:53.329 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T01:06:53.347 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T01:06:53.390 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T01:06:53.582 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T01:06:53.609 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T01:06:53.648 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T01:06:53.746 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T01:06:53.775 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T01:06:53.813 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T01:06:53.952 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T01:06:53.979 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T01:06:54.010 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T01:06:54.209 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T01:06:54.238 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T01:06:54.262 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T01:06:54.275 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T01:06:54.278 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T01:06:54.321 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T01:06:54.557 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T01:06:54.573 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T01:06:54.615 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T01:06:54.737 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T01:06:54.766 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T01:06:54.820 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T01:06:54.926 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T01:06:54.954 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T01:06:54.993 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T01:06:55.135 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T01:06:55.361 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T01:06:55.462 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T01:06:55.588 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:06:55.714 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T01:06:55.840 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T01:06:55.966 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:06:56.524 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T01:06:56.614 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T01:06:56.688 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T01:06:56.715 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T01:06:56.754 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:06:57.000 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T01:06:57.000 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T01:06:57.015 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T01:06:57.031 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T01:06:57.073 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:06:57.418 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T01:06:57.445 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T01:06:57.484 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:06:57.500 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T01:06:57.634 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T01:06:57.752 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T01:06:57.796 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T01:06:57.820 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T01:06:57.853 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T01:06:57.862 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T01:06:57.954 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T01:06:58.113 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T01:06:58.149 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T01:06:58.165 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T01:06:58.198 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T01:06:58.239 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T01:06:58.365 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T01:06:58.526 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T01:06:58.555 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:58.593 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:06:58.930 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T01:06:58.957 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:58.963 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T01:06:58.996 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:06:59.089 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T01:06:59.182 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T01:06:59.249 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:06:59.332 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T01:06:59.360 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:06:59.375 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T01:06:59.398 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:06:59.510 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T01:06:59.627 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T01:06:59.643 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T01:06:59.659 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T01:06:59.693 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T01:06:59.728 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T01:06:59.809 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T01:06:59.871 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T01:06:59.997 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T01:07:00.174 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T01:07:00.188 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T01:07:00.216 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T01:07:00.254 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T01:07:00.566 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package modemmanager. 2024-04-25T01:07:00.593 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T01:07:00.632 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:07:00.667 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:07:00.785 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T01:07:00.919 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T01:07:01.020 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T01:07:01.306 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package network-manager. 2024-04-25T01:07:01.334 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T01:07:01.380 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T01:07:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:01.494 INFO:teuthology.orchestra.run.smithi033.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T01:07:01.494 INFO:teuthology.orchestra.run.smithi033.stdout:which means they are currently configured by ifupdown: 2024-04-25T01:07:01.494 INFO:teuthology.orchestra.run.smithi033.stdout:- ens1f0 2024-04-25T01:07:01.494 INFO:teuthology.orchestra.run.smithi033.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T01:07:01.494 INFO:teuthology.orchestra.run.smithi033.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T01:07:01.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:01.616 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T01:07:01.905 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package ppp. 2024-04-25T01:07:01.933 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T01:07:01.935 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T01:07:02.211 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T01:07:02.253 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T01:07:02.518 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T01:07:02.534 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T01:07:02.576 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T01:07:02.714 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:07:02.838 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T01:07:02.849 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T01:07:02.866 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T01:07:02.921 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T01:07:02.975 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T01:07:03.101 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up postgresql-common (238) ... 2024-04-25T01:07:03.224 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T01:07:03.252 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T01:07:03.290 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T01:07:03.577 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T01:07:03.594 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:07:03.636 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:07:03.913 INFO:teuthology.orchestra.run.smithi033.stdout:Adding user postgres to group ssl-cert 2024-04-25T01:07:04.122 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:04.123 INFO:teuthology.orchestra.run.smithi033.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T01:07:04.224 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T01:07:04.252 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T01:07:04.308 INFO:teuthology.orchestra.run.smithi138.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T01:07:04.311 INFO:teuthology.orchestra.run.smithi033.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T01:07:04.311 INFO:teuthology.orchestra.run.smithi033.stdout:Removing obsolete dictionary files: 2024-04-25T01:07:04.338 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-common (238) ... 2024-04-25T01:07:04.814 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T01:07:04.842 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:07:04.860 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T01:07:04.884 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:07:05.245 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:07:05.619 INFO:teuthology.orchestra.run.smithi033.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T01:07:05.639 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T01:07:05.679 INFO:teuthology.orchestra.run.smithi033.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T01:07:05.679 INFO:teuthology.orchestra.run.smithi033.stdout:This user must also own the server process. 2024-04-25T01:07:05.679 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:05.679 INFO:teuthology.orchestra.run.smithi033.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T01:07:05.679 INFO:teuthology.orchestra.run.smithi033.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T01:07:05.679 INFO:teuthology.orchestra.run.smithi033.stdout:The default text search configuration will be set to "english". 2024-04-25T01:07:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:05.680 INFO:teuthology.orchestra.run.smithi033.stdout:Data page checksums are disabled. 2024-04-25T01:07:05.680 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:05.680 INFO:teuthology.orchestra.run.smithi033.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T01:07:05.680 INFO:teuthology.orchestra.run.smithi033.stdout:creating subdirectories ... ok 2024-04-25T01:07:05.680 INFO:teuthology.orchestra.run.smithi033.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T01:07:05.693 INFO:teuthology.orchestra.run.smithi033.stdout:selecting default max_connections ... 100 2024-04-25T01:07:05.712 INFO:teuthology.orchestra.run.smithi033.stdout:selecting default shared_buffers ... 128MB 2024-04-25T01:07:05.722 INFO:teuthology.orchestra.run.smithi033.stdout:selecting default time zone ... Etc/UTC 2024-04-25T01:07:05.723 INFO:teuthology.orchestra.run.smithi033.stdout:creating configuration files ... ok 2024-04-25T01:07:05.836 INFO:teuthology.orchestra.run.smithi033.stdout:running bootstrap script ... ok 2024-04-25T01:07:06.242 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql. 2024-04-25T01:07:06.258 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T01:07:06.300 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql (14+238) ... 2024-04-25T01:07:06.486 INFO:teuthology.orchestra.run.smithi033.stdout:performing post-bootstrap initialization ... ok 2024-04-25T01:07:06.544 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T01:07:06.560 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T01:07:06.602 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T01:07:06.854 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T01:07:06.870 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T01:07:06.913 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T01:07:07.081 INFO:teuthology.orchestra.run.smithi033.stdout:syncing data to disk ... ok 2024-04-25T01:07:07.207 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T01:07:07.235 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T01:07:07.265 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T01:07:07.459 INFO:teuthology.orchestra.run.smithi033.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T01:07:07.534 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package tcl. 2024-04-25T01:07:07.563 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T01:07:07.606 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T01:07:07.963 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T01:07:07.991 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T01:07:08.029 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T01:07:08.391 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T01:07:08.407 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T01:07:08.449 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T01:07:08.794 INFO:teuthology.orchestra.run.smithi138.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T01:07:08.823 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T01:07:09.077 INFO:teuthology.orchestra.run.smithi138.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T01:07:09.465 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T01:07:09.675 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T01:07:09.793 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T01:07:09.902 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:07:10.028 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T01:07:10.154 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T01:07:10.219 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:07:10.272 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:07:10.328 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T01:07:10.454 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up postgresql (14+238) ... 2024-04-25T01:07:10.580 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:07:10.706 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:07:10.796 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T01:07:10.886 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T01:07:10.967 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T01:07:10.967 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T01:07:11.509 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T01:07:11.610 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T01:07:11.822 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T01:07:11.912 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T01:07:11.948 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T01:07:12.072 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T01:07:12.074 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T01:07:12.200 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T01:07:12.318 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T01:07:12.427 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T01:07:12.553 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T01:07:12.679 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T01:07:13.451 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T01:07:13.476 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T01:07:13.575 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T01:07:13.577 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T01:07:13.671 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T01:07:13.737 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:07:13.863 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T01:07:13.981 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T01:07:14.082 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T01:07:14.208 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T01:07:14.263 INFO:teuthology.orchestra.run.smithi033.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T01:07:14.269 DEBUG:teuthology.parallel:result is None 2024-04-25T01:07:14.297 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T01:07:14.367 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T01:07:14.494 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T01:07:14.670 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T01:07:14.796 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:07:14.922 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T01:07:15.082 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T01:07:15.182 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T01:07:15.681 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:15.681 INFO:teuthology.orchestra.run.smithi138.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T01:07:15.681 INFO:teuthology.orchestra.run.smithi138.stdout:which means they are currently configured by ifupdown: 2024-04-25T01:07:15.682 INFO:teuthology.orchestra.run.smithi138.stdout:- enp3s0f1 2024-04-25T01:07:15.682 INFO:teuthology.orchestra.run.smithi138.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T01:07:15.682 INFO:teuthology.orchestra.run.smithi138.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T01:07:15.682 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:15.797 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T01:07:16.116 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T01:07:16.449 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T01:07:16.968 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T01:07:17.120 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T01:07:17.246 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T01:07:17.372 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-common (238) ... 2024-04-25T01:07:18.264 INFO:teuthology.orchestra.run.smithi138.stdout:Adding user postgres to group ssl-cert 2024-04-25T01:07:18.487 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:18.487 INFO:teuthology.orchestra.run.smithi138.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T01:07:18.678 INFO:teuthology.orchestra.run.smithi138.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T01:07:18.678 INFO:teuthology.orchestra.run.smithi138.stdout:Removing obsolete dictionary files: 2024-04-25T01:07:19.250 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T01:07:19.648 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T01:07:19.968 INFO:teuthology.orchestra.run.smithi138.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T01:07:19.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:This user must also own the server process. 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:The default text search configuration will be set to "english". 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:Data page checksums are disabled. 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:creating subdirectories ... ok 2024-04-25T01:07:20.041 INFO:teuthology.orchestra.run.smithi138.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T01:07:20.056 INFO:teuthology.orchestra.run.smithi138.stdout:selecting default max_connections ... 100 2024-04-25T01:07:20.075 INFO:teuthology.orchestra.run.smithi138.stdout:selecting default shared_buffers ... 128MB 2024-04-25T01:07:20.084 INFO:teuthology.orchestra.run.smithi138.stdout:selecting default time zone ... Etc/UTC 2024-04-25T01:07:20.084 INFO:teuthology.orchestra.run.smithi138.stdout:creating configuration files ... ok 2024-04-25T01:07:20.197 INFO:teuthology.orchestra.run.smithi138.stdout:running bootstrap script ... ok 2024-04-25T01:07:20.874 INFO:teuthology.orchestra.run.smithi138.stdout:performing post-bootstrap initialization ... ok 2024-04-25T01:07:21.568 INFO:teuthology.orchestra.run.smithi138.stdout:syncing data to disk ... ok 2024-04-25T01:07:21.975 INFO:teuthology.orchestra.run.smithi138.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T01:07:24.813 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:07:24.930 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T01:07:25.040 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up postgresql (14+238) ... 2024-04-25T01:07:25.157 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T01:07:25.283 INFO:teuthology.orchestra.run.smithi138.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T01:07:25.539 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T01:07:25.539 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T01:07:26.136 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T01:07:26.228 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T01:07:26.522 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T01:07:26.674 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T01:07:28.027 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T01:07:28.118 INFO:teuthology.orchestra.run.smithi138.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T01:07:28.602 INFO:teuthology.orchestra.run.smithi138.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T01:07:28.607 DEBUG:teuthology.parallel:result is None 2024-04-25T01:07:28.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:07:28.832 DEBUG:teuthology.orchestra.run.smithi033:> dpkg-query -W -f '${Version}' ceph 2024-04-25T01:07:28.852 INFO:teuthology.orchestra.run.smithi033.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:07:28.853 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:07:28.853 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T01:07:28.855 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:07:28.990 DEBUG:teuthology.orchestra.run.smithi138:> dpkg-query -W -f '${Version}' ceph 2024-04-25T01:07:29.009 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:07:29.009 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445-gd2c45d97-1jammy 2024-04-25T01:07:29.009 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445-gd2c45d97-1jammy is installed. 2024-04-25T01:07:29.011 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T01:07:29.011 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:29.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:07:29.026 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:29.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:07:29.062 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T01:07:29.062 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:29.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:07:29.081 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:07:29.136 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:29.136 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:07:29.149 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:07:29.204 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T01:07:29.205 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:29.205 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:07:29.218 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:07:29.272 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:29.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:07:29.285 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:07:29.337 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T01:07:29.337 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:29.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:07:29.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:07:29.404 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:29.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:07:29.417 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:07:29.473 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T01:07:29.560 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T01:07:29.560 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2024-04-25T01:07:29.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-04-25T01:07:29.576 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T01:07:29.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T01:07:29.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T01:07:29.630 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T01:07:29.630 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T01:07:29.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T01:07:29.692 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T01:07:29.693 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T01:07:29.693 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T01:07:29.693 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:29.693 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T01:07:29.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T01:07:29.811 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:29.811 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T01:07:29.817 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T01:07:29.892 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T01:07:29.892 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T01:07:29.893 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7672298', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8528, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '122/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi033.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0HvAv8M2hCk3E5ruY79aT3zBJNLSOaRGctm1CEq9ALA9tfcqfEXFdS8Yun7cainNS4pwa5ZFY1I3fea1XPbOM=', 'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHaEzxCOKVFgS3Fm1PlIYxXr3G+iem+4DYLvPFbtwTmW0Xw0RffLfUajXYHhN6uHJtu88OK+vnx44TjqX91ZoW8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_acls']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_21:24:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T01:07:29.893 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T01:07:29.896 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T01:07:29.934 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T01:07:29.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T01:07:29.991 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:29.991 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:07:30.001 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:07:30.001 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-04-25T01:07:30.049 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:07:30.050 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:30.050 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 871 Links: 1 2024-04-25T01:07:30.050 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:30.050 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:00:49.141237626 +0000 2024-04-25T01:07:30.050 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:00:48.901239770 +0000 2024-04-25T01:07:30.050 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:00:48.901239770 +0000 2024-04-25T01:07:30.050 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-04-25T01:07:30.050 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:07:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:07:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:07:30.104 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000378559 s, 1.4 MB/s 2024-04-25T01:07:30.105 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:07:30.154 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-04-25T01:07:30.201 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:07:30.201 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:30.201 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-04-25T01:07:30.201 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:30.202 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:00:49.637233196 +0000 2024-04-25T01:07:30.202 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:00:49.365235625 +0000 2024-04-25T01:07:30.202 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:00:49.365235625 +0000 2024-04-25T01:07:30.202 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-04-25T01:07:30.202 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:07:30.255 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:07:30.255 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:07:30.255 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000310923 s, 1.6 MB/s 2024-04-25T01:07:30.256 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:07:30.306 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-04-25T01:07:30.353 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:07:30.353 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:30.353 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-25T01:07:30.353 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:30.354 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:00:50.173228408 +0000 2024-04-25T01:07:30.354 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:00:49.917230694 +0000 2024-04-25T01:07:30.354 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:00:49.917230694 +0000 2024-04-25T01:07:30.354 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-04-25T01:07:30.354 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:07:30.407 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:07:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:07:30.408 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000356822 s, 1.4 MB/s 2024-04-25T01:07:30.409 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:07:30.459 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-04-25T01:07:30.505 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:07:30.506 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:30.506 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-04-25T01:07:30.506 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:30.506 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:00:50.685223835 +0000 2024-04-25T01:07:30.506 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:00:50.421226193 +0000 2024-04-25T01:07:30.506 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:00:50.421226193 +0000 2024-04-25T01:07:30.506 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-04-25T01:07:30.506 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:07:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:07:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:07:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000281599 s, 1.8 MB/s 2024-04-25T01:07:30.561 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:07:30.610 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:30.610 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:30.610 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:07:30.616 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:07:30.616 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:00:49.179576829 +0000 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:00:48.895579365 +0000 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:00:48.895579365 +0000 2024-04-25T01:07:30.663 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T01:07:30.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:07:30.717 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:07:30.717 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:07:30.717 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00045238 s, 1.1 MB/s 2024-04-25T01:07:30.719 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:07:30.768 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-25T01:07:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:07:30.814 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:30.814 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-04-25T01:07:30.815 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:30.815 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:00:49.407574792 +0000 2024-04-25T01:07:30.815 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:00:49.403574828 +0000 2024-04-25T01:07:30.815 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:00:49.403574828 +0000 2024-04-25T01:07:30.815 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T01:07:30.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:07:30.868 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:07:30.868 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:07:30.868 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000332465 s, 1.5 MB/s 2024-04-25T01:07:30.870 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:07:30.920 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-25T01:07:30.966 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:07:30.966 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:30.966 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 911 Links: 1 2024-04-25T01:07:30.966 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:30.967 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:00:49.915570254 +0000 2024-04-25T01:07:30.967 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:00:49.911570290 +0000 2024-04-25T01:07:30.967 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:00:49.911570290 +0000 2024-04-25T01:07:30.967 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T01:07:30.967 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:07:31.020 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:07:31.020 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:07:31.020 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000308776 s, 1.7 MB/s 2024-04-25T01:07:31.021 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:07:31.072 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:00:50.683563395 +0000 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:00:50.411565824 +0000 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:00:50.411565824 +0000 2024-04-25T01:07:31.119 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T01:07:31.120 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:07:31.173 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:07:31.173 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:07:31.173 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000392554 s, 1.3 MB/s 2024-04-25T01:07:31.174 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:07:31.223 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:31.224 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T01:07:31.224 INFO:tasks.ceph:Generating config... 2024-04-25T01:07:31.225 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T01:07:31.225 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T01:07:31.225 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T01:07:31.225 INFO:tasks.ceph:[client] pid file = 2024-04-25T01:07:31.225 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T01:07:31.225 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T01:07:31.225 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T01:07:31.225 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T01:07:31.225 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T01:07:31.225 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T01:07:31.225 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T01:07:31.226 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T01:07:31.226 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T01:07:31.226 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T01:07:31.227 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T01:07:31.227 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T01:07:31.228 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T01:07:31.262 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T01:07:31.278 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T01:07:31.358 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:07:31.412 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.b', '172.21.15.138'), ('mon.c', '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]')] 2024-04-25T01:07:31.413 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.33,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T01:07:31.414 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:31.414 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T01:07:31.461 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.33 --add b 172.21.15.138 --addv c '[v2:172.21.15.138:3301,v1:172.21.15.138:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:31.549 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:31.549 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid 10df718f-e6ff-4eb1-9854-6e8a751898ff 2024-04-25T01:07:31.549 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-04-25T01:07:31.549 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:fsid 10df718f-e6ff-4eb1-9854-6e8a751898ff 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2024-04-25T01:07:31.559252+0000 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:07:31.559252+0000 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 17 (quincy) 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:election_strategy: 1 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.b 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] mon.c 2024-04-25T01:07:31.550 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T01:07:31.552 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T01:07:31.557 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 10df718f-e6ff-4eb1-9854-6e8a751898ff... 2024-04-25T01:07:31.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T01:07:31.602 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T01:07:31.630 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-04-25T01:07:31.630 INFO:teuthology.orchestra.run.smithi033.stdout: chdir = "" 2024-04-25T01:07:31.630 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: auth supported = cephx 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: filestore xattr use omap = true 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: mon clock drift allowed = 1.000 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: osd crush chooseleaf type = 0 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: auth debug = true 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on old message = true 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: ms die on bug = true 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T01:07:31.631 INFO:teuthology.orchestra.run.smithi033.stdout: mon pg warn max object skew = 0 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default size = 2 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow primary affinity = true 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd allow pg remap = true 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on legacy crush tunables = false 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on crush straw calc version zero = false 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on no sortbitwise = false 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on osd down out interval zero = false 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon warn on too few osds = false 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T01:07:31.632 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: mon_allow_pool_size_one = true 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: osd default data pool replay window = 5 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: mon allow pool delete = true 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: mon cluster log file level = debug 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: debug asserts on shutdown = true 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: mon health detail to clog = false 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: mon host = "172.21.15.33,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = 10df718f-e6ff-4eb1-9854-6e8a751898ff 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.633 INFO:teuthology.orchestra.run.smithi033.stdout:[osd] 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd journal size = 100 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub load threshold = 5.0 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd scrub max interval = 600 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd mclock profile = high_recovery_ops 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd recover clone overlap = true 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd recovery max chunk = 1048576 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug shutdown = true 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug op order = true 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug verify stray on activate = true 2024-04-25T01:07:31.634 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug trim objects = true 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: osd open classes on start = true 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug pg log writeout = true 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: osd deep scrub update digest min age = 30 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: osd map max advance = 10 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: journal zero on create = true 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: filestore ondisk finisher threads = 3 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: filestore apply finisher threads = 3 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: bdev debug aio = true 2024-04-25T01:07:31.635 INFO:teuthology.orchestra.run.smithi033.stdout: osd debug misdirected ops = true 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore block size = 96636764160 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: bluestore fsck on mount = True 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluefs = 20 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: debug bluestore = 20 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: debug osd = 20 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: debug rocksdb = 10 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd full ratio = 0.9 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd nearfull ratio = 0.8 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: osd failsafe full ratio = 0.95 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: osd objectstore = bluestore 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr] 2024-04-25T01:07:31.636 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug mgr = 20 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: mgr/telemetry/nag = false 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout:[mon] 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug mon = 20 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug paxos = 20 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: debug auth = 20 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: mon data avail warn = 5 2024-04-25T01:07:31.637 INFO:teuthology.orchestra.run.smithi033.stdout: mon mgr mkfs grace = 240 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd reporter subtree level = osd 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: mon osd prime pg temp = true 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T01:07:31.638 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: # 1m isn't quite enough 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: mon_down_mkfs_grace = 2m 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: mon_warn_on_filestore_osds = false 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: mon down mkfs grace = 300 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: mon op complaint time = 120 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout:[client] 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: rgw cache enabled = true 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable ops log = true 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: rgw enable usage log = true 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T01:07:31.639 INFO:teuthology.orchestra.run.smithi033.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout: client mount timeout = 600 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout: debug client = 20 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout: pid file = "" 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.a] 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.b] 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout:[mon.c] 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2024-04-25T01:07:31.640 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout:[mds] 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds = 20 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: debug mds balancer = 20 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: debug ms = 1 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug frag = True 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: mds debug scatterstat = True 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: mds op complaint time = 180 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: mds verify scatter = True 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: osd op complaint time = 180 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: rados mon op timeout = 900 2024-04-25T01:07:31.641 INFO:teuthology.orchestra.run.smithi033.stdout: rados osd op timeout = 900 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.642 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.643 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-04-25T01:07:31.644 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.33,172.21.15.138,[v2:172.21.15.138:3301,v1:172.21.15.138:6790]" 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = 10df718f-e6ff-4eb1-9854-6e8a751898ff 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-04-25T01:07:31.645 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.646 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 20 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 20 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-04-25T01:07:31.647 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 10 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-25T01:07:31.648 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-04-25T01:07:31.649 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-04-25T01:07:31.650 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: mon down mkfs grace = 300 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-04-25T01:07:31.651 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = "" 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-04-25T01:07:31.652 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-04-25T01:07:31.653 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-04-25T01:07:31.653 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-04-25T01:07:31.653 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-04-25T01:07:31.653 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-04-25T01:07:31.653 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-04-25T01:07:31.653 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-04-25T01:07:31.653 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-04-25T01:07:31.654 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T01:07:31.654 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T01:07:31.745 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T01:07:31.745 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:31.745 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T01:07:31.750 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:31.750 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T01:07:31.796 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:07:31.797 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:31.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T01:07:31.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:07:31.864 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:07:31.864 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:31.910 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:07:31.910 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:31.910 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T01:07:31.910 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T01:07:31.934 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:07:31.934 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:31.982 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T01:07:31.982 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T01:07:31.983 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T01:07:32.027 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T01:07:32.030 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T01:07:32.074 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T01:07:32.093 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T01:07:32.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T01:07:32.135 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T01:07:32.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T01:07:32.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T01:07:32.277 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T01:07:32.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T01:07:32.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T01:07:32.382 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T01:07:32.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T01:07:32.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T01:07:32.479 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T01:07:32.482 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T01:07:32.537 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T01:07:32.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T01:07:32.575 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T01:07:32.586 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T01:07:32.662 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T01:07:32.673 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T01:07:32.709 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T01:07:32.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T01:07:32.791 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T01:07:32.801 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T01:07:32.802 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T01:07:32.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T01:07:32.813 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:32.813 INFO:tasks.ceph:role: osd.0 2024-04-25T01:07:32.814 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:07:32.814 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T01:07:32.877 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:32.877 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:32.877 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:32.877 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:32.877 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:32.877 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:32.878 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:32.878 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:32.878 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:32.878 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:33.336 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-04-25T01:07:33.338 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-04-25T01:07:33.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T01:07:33.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T01:07:33.448 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:33.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:33.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T01:07:33.502 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:33.502 INFO:tasks.ceph:role: osd.1 2024-04-25T01:07:33.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:07:33.503 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T01:07:33.564 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:33.564 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:33.564 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:33.565 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:33.565 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:33.565 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:33.565 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:33.565 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:33.565 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:33.565 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:34.027 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-04-25T01:07:34.029 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-04-25T01:07:34.029 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T01:07:34.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T01:07:34.182 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:34.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:34.183 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T01:07:34.237 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:34.237 INFO:tasks.ceph:role: osd.2 2024-04-25T01:07:34.237 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:07:34.237 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T01:07:34.300 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:34.300 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:34.300 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:34.301 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:34.301 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:34.301 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:34.301 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:34.301 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:34.301 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:34.301 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:34.787 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-04-25T01:07:34.790 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-04-25T01:07:34.790 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T01:07:34.939 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T01:07:34.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:34.954 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:34.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T01:07:35.009 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:35.009 INFO:tasks.ceph:role: osd.3 2024-04-25T01:07:35.009 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:07:35.009 DEBUG:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:35.075 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:35.076 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:35.076 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:Discarding blocks...Done. 2024-04-25T01:07:35.498 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2024-04-25T01:07:35.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T01:07:35.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T01:07:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:35.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:35.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:35.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:35.727+0000 7f0215fbe7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T01:07:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:35.727+0000 7f0215fbe7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T01:07:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:35.727+0000 7f0215fbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:07:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:35.727+0000 7f0215fbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:07:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:35.727+0000 7f0215fbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T01:07:35.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:35.727+0000 7f0215fbe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T01:07:38.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T01:07:38.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:38.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:38.827+0000 7ff544af57c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T01:07:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:38.831+0000 7ff544af57c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T01:07:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:38.831+0000 7ff544af57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:07:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:38.831+0000 7ff544af57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:07:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:38.831+0000 7ff544af57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T01:07:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:38.831+0000 7ff544af57c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T01:07:41.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T01:07:41.894 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:41.979+0000 7facfbc427c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T01:07:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:41.983+0000 7facfbc427c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T01:07:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:41.983+0000 7facfbc427c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:07:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:41.983+0000 7facfbc427c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:07:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:41.983+0000 7facfbc427c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T01:07:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:41.983+0000 7facfbc427c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T01:07:44.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T01:07:44.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:45.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:45.091+0000 7fb2ee17e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T01:07:45.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:45.091+0000 7fb2ee17e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T01:07:45.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:45.091+0000 7fb2ee17e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:07:45.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:45.091+0000 7fb2ee17e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:07:45.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:45.091+0000 7fb2ee17e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T01:07:45.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:07:45.091+0000 7fb2ee17e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T01:07:48.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T01:07:48.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T01:07:48.152 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:48.152 INFO:tasks.ceph:role: osd.4 2024-04-25T01:07:48.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:07:48.152 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T01:07:48.214 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:48.215 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:48.661 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T01:07:48.663 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T01:07:48.663 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T01:07:48.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T01:07:48.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:48.767 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:48.767 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T01:07:48.822 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:48.822 INFO:tasks.ceph:role: osd.5 2024-04-25T01:07:48.822 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:07:48.822 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:48.887 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:49.354 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T01:07:49.356 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T01:07:49.356 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T01:07:49.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T01:07:49.495 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:49.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:49.496 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T01:07:49.549 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:49.550 INFO:tasks.ceph:role: osd.6 2024-04-25T01:07:49.550 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:07:49.550 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T01:07:49.614 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:49.614 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:49.614 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:49.615 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:49.615 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:49.615 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:49.615 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:50.090 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T01:07:50.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T01:07:50.092 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T01:07:50.213 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T01:07:50.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:50.224 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:50.225 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T01:07:50.278 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T01:07:50.278 INFO:tasks.ceph:role: osd.7 2024-04-25T01:07:50.278 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:07:50.278 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T01:07:50.342 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T01:07:50.343 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T01:07:50.772 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-04-25T01:07:50.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-04-25T01:07:50.775 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T01:07:50.857 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T01:07:50.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:07:50.911 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T01:07:50.911 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:51.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:51.039+0000 7fdad9f1e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T01:07:51.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:51.043+0000 7fdad9f1e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T01:07:51.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:51.043+0000 7fdad9f1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:07:51.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:51.043+0000 7fdad9f1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:07:51.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:51.043+0000 7fdad9f1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T01:07:51.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:51.043+0000 7fdad9f1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T01:07:54.011 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T01:07:54.025 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:54.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:54.115+0000 7ff20750e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T01:07:54.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:54.119+0000 7ff20750e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T01:07:54.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:54.119+0000 7ff20750e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:07:54.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:54.119+0000 7ff20750e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:07:54.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:54.119+0000 7ff20750e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T01:07:54.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:54.119+0000 7ff20750e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T01:07:57.073 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T01:07:57.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:07:57.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:57.219+0000 7fc3c81967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T01:07:57.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:57.219+0000 7fc3c81967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T01:07:57.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:57.219+0000 7fc3c81967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:07:57.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:57.219+0000 7fc3c81967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:07:57.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:57.219+0000 7fc3c81967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T01:07:57.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:07:57.219+0000 7fc3c81967c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T01:07:59.895 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:07:59.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:08:00.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T01:08:00.209 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:08:00.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:00.299+0000 7f52a3adc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T01:08:00.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:00.299+0000 7f52a3adc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T01:08:00.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:00.299+0000 7f52a3adc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:08:00.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:00.299+0000 7f52a3adc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:08:00.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:00.299+0000 7f52a3adc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T01:08:00.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:00.299+0000 7f52a3adc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T01:08:03.248 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T01:08:03.302 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T01:08:03.303 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T01:08:03.317 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.317 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T01:08:03.373 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.373 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T01:08:03.429 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T01:08:03.484 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.484 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T01:08:03.540 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.540 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T01:08:03.596 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T01:08:03.653 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:03.653 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T01:08:03.667 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:03.667 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T01:08:03.722 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:03.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T01:08:03.777 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:03.778 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T01:08:03.834 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:03.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T01:08:03.889 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:03.889 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T01:08:03.941 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:03.941 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T01:08:03.997 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:03.997 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T01:08:04.003 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:04.003 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T01:08:04.049 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:04.049 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T01:08:04.054 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:04.054 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T01:08:04.103 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T01:08:04.103 DEBUG:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T01:08:04.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T01:08:04.148 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2024-04-25T01:08:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmG2U4AhAAJWFzFjoGGiuamFr4u+FWeA== 2024-04-25T01:08:04.148 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a] 2024-04-25T01:08:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClm/1anCBAAAd5rluwqED/HsqyQXnS4bw== 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c] 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmadkeERAAWiEIDRFoI3UFC0gmjTUCzg== 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDXrClmO46RKxAA9f2l5EWxqEUjAeg2/dZ7BQ== 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDarClmjBi6MRAA1HxP2UL7163J1a6EatsCrQ== 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDdrClmucXKOhAAaenJnSHXygXuJw9D3uiSIg== 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDhrClmHO+kBRAAi2Mtq+PLtDWRZtv+Jb+HnA== 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmoJwYBRAANlFwhdQApqMEg1FRrFyLGA== 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b] 2024-04-25T01:08:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmceVqFxAADWwU51ebqTMdk9oRKfRyqw== 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d] 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmzuw8HRAArbWjoWoWc9QnmClOslRhyA== 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDnrClmORmeAhAAf5kNhiCpkd0oJAVYReRnfw== 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDqrClm0zMpBxAAx9DWUbvSD29vy0aj0KbphQ== 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDtrClm26g+DRAAgR+9igOsmcAO4/cYVfW40g== 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDwrClmyBEIEhAAZT4pGOBNnrffGisaecuD0g== 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmzPTbIhAAvzORiMqrE/Y+uTLvKO5BMw== 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout:[client.2] 2024-04-25T01:08:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClme7kQKBAAtTHHa+QxjysQ5Gzb8+DNLw== 2024-04-25T01:08:04.151 INFO:teuthology.orchestra.run.smithi033.stdout:[client.1] 2024-04-25T01:08:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmegPyKhAAHLA/5dqn09dPyHPr/X6z3w== 2024-04-25T01:08:04.151 INFO:teuthology.orchestra.run.smithi033.stdout:[client.3] 2024-04-25T01:08:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmzPDQLxAApaIke8y8GYt0E2+ZODUjiA== 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmG2U4AhAAJWFzFjoGGiuamFr4u+FWeA== 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClm/1anCBAAAd5rluwqED/HsqyQXnS4bw== 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmadkeERAAWiEIDRFoI3UFC0gmjTUCzg== 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDXrClmO46RKxAA9f2l5EWxqEUjAeg2/dZ7BQ== 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDarClmjBi6MRAA1HxP2UL7163J1a6EatsCrQ== 2024-04-25T01:08:04.157 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDdrClmucXKOhAAaenJnSHXygXuJw9D3uiSIg== 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDhrClmHO+kBRAAi2Mtq+PLtDWRZtv+Jb+HnA== 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmoJwYBRAANlFwhdQApqMEg1FRrFyLGA== 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmceVqFxAADWwU51ebqTMdk9oRKfRyqw== 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmzuw8HRAArbWjoWoWc9QnmClOslRhyA== 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDnrClmORmeAhAAf5kNhiCpkd0oJAVYReRnfw== 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDqrClm0zMpBxAAx9DWUbvSD29vy0aj0KbphQ== 2024-04-25T01:08:04.158 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDtrClm26g+DRAAgR+9igOsmcAO4/cYVfW40g== 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDwrClmyBEIEhAAZT4pGOBNnrffGisaecuD0g== 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmzPTbIhAAvzORiMqrE/Y+uTLvKO5BMw== 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout:[client.2] 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClme7kQKBAAtTHHa+QxjysQ5Gzb8+DNLw== 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmegPyKhAAHLA/5dqn09dPyHPr/X6z3w== 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout:[client.3] 2024-04-25T01:08:04.159 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQDUrClmzPDQLxAApaIke8y8GYt0E2+ZODUjiA== 2024-04-25T01:08:04.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:08:04.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:08:04.248 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.297 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.390 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.432 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.477 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.562 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.646 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:08:04.691 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T01:08:04.772 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.774 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.861 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T01:08:04.909 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.950 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:04.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:05.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:05.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:05.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:05.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:05.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T01:08:05.258 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.260 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.351 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T01:08:05.531 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T01:08:05.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T01:08:05.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:08:05.668 DEBUG:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T01:08:05.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T01:08:05.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:08:05.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T01:08:05.840 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T01:08:05.891 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T01:08:05.979 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T01:08:05.990 DEBUG:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:08:05.992 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T01:08:06.038 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T01:08:06.038 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T01:08:06.038 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T01:08:06.041 INFO:tasks.ceph.mon.a:Started 2024-04-25T01:08:06.041 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T01:08:06.041 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T01:08:06.083 INFO:tasks.ceph.mon.b:Started 2024-04-25T01:08:06.083 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T01:08:06.084 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T01:08:06.085 INFO:tasks.ceph.mon.c:Started 2024-04-25T01:08:06.086 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T01:08:06.086 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T01:08:06.086 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T01:08:06.089 INFO:tasks.ceph.mgr.y:Started 2024-04-25T01:08:06.089 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T01:08:06.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T01:08:06.091 INFO:tasks.ceph.mgr.x:Started 2024-04-25T01:08:06.091 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T01:08:06.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T01:08:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f113612e640 1 Processor -- start 2024-04-25T01:08:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f113612e640 1 -- start start 2024-04-25T01:08:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f113612e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f113612e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11300b6cc0 con 0x7f113015e0d0 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f113612e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11300b6f30 con 0x7f11300b9a20 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f113612e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11300cfb00 con 0x7f11300d1f60 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f112f7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f113015e0d0 0x7f11300cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:60960/0 (socket says 172.21.15.33:60960) 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.210+0000 7f112f7fe640 1 -- 172.21.15.33:0/923309016 learned_addr learned my addr 172.21.15.33:0/923309016 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.214+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.214+0000 7f112effd640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 msgr2=0x7f11300d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.214+0000 7f112effd640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 msgr2=0x7f11300d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.214+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:08:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.214+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:08:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.418+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.418+0000 7f112effd640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 msgr2=0x7f11300d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T01:08:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.418+0000 7f112effd640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 msgr2=0x7f11300d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T01:08:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.418+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T01:08:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.418+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:08:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1057783044 0 0) 0x7f11300b6cc0 con 0x7f113015e0d0 2024-04-25T01:08:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1118003530 con 0x7f113015e0d0 2024-04-25T01:08:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1626185145 0 0) 0x7f11300b6f30 con 0x7f11300b9a20 2024-04-25T01:08:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11300b6cc0 con 0x7f11300b9a20 2024-04-25T01:08:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1747722607 0 0) 0x7f1118003530 con 0x7f113015e0d0 2024-04-25T01:08:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 msgr2=0x7f11300d23b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:08:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f11300d23b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 >> v1:172.21.15.138:6789/0 conn(0x7f11300b9a20 legacy=0x7f1130167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11300b7210 con 0x7f113015e0d0 2024-04-25T01:08:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.454+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f1120001680 con 0x7f113015e0d0 2024-04-25T01:08:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 >> v1:172.21.15.33:6789/0 conn(0x7f113015e0d0 legacy=0x7f11300cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112e7fc640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f1118003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112e7fc640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11180040e0 0x7f1118004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112e7fc640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1118004d50 0x7f11180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:06.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11300cfb00 con 0x7f11180040e0 2024-04-25T01:08:06.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f111800bb60 con 0x7f1118004d50 2024-04-25T01:08:06.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f111800bd60 con 0x7f11300d1f60 2024-04-25T01:08:06.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11180040e0 0x7f1118004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112f7fe640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f1118003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112ffff640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1118004d50 0x7f11180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112effd640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 msgr2=0x7f1118003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:06.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f1118003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112effd640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1118004d50 msgr2=0x7f11180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:06.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1118004d50 0x7f11180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112effd640 1 -- 172.21.15.33:0/923309016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1118003530 con 0x7f11180040e0 2024-04-25T01:08:06.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112f7fe640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11300d1f60 0x7f1118003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:06.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.458+0000 7f112ffff640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1118004d50 0x7f11180051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:06.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.462+0000 7f112effd640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11180040e0 0x7f1118004810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11300d2360 tx=0x7f112401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d547ab9ad38b5711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:06.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240ab070 con 0x7f11180040e0 2024-04-25T01:08:06.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11240a5710 con 0x7f11180040e0 2024-04-25T01:08:06.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f112e7fc640 1 -- 172.21.15.33:0/923309016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112400f040 con 0x7f11180040e0 2024-04-25T01:08:06.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f113612e640 1 -- 172.21.15.33:0/923309016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11180040e0 msgr2=0x7f1118004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:06.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f113612e640 1 --2- 172.21.15.33:0/923309016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11180040e0 0x7f1118004810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f11300d2360 tx=0x7f112401c810 comp rx=0 tx=0).stop 2024-04-25T01:08:06.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f112ffff640 1 -- 172.21.15.33:0/923309016 reap_dead start 2024-04-25T01:08:06.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f113612e640 1 -- 172.21.15.33:0/923309016 shutdown_connections 2024-04-25T01:08:06.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.466+0000 7f113612e640 1 -- 172.21.15.33:0/923309016 >> 172.21.15.33:0/923309016 conn(0x7f11300d3dd0 msgr2=0x7f11300d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:06.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.470+0000 7f113612e640 1 -- 172.21.15.33:0/923309016 shutdown_connections 2024-04-25T01:08:06.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.470+0000 7f113612e640 1 -- 172.21.15.33:0/923309016 wait complete. 2024-04-25T01:08:06.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.470+0000 7f113612e640 1 Processor -- start 2024-04-25T01:08:06.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.470+0000 7f113612e640 1 -- start start 2024-04-25T01:08:06.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.470+0000 7f113612e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11180040e0 0x7f11301f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:06.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.470+0000 7f113612e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11301511a0 0x7f113015f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:06.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f113612e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f113015fd40 0x7f11300c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:06.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f113612e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f113015e4b0 con 0x7f11301511a0 2024-04-25T01:08:06.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f113612e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11300b7580 con 0x7f11180040e0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f113612e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f113015e630 con 0x7f113015fd40 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11301511a0 0x7f113015f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11301511a0 0x7f113015f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34482/0 (socket says 172.21.15.33:34482) 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 -- 172.21.15.33:0/19431397 learned_addr learned my addr 172.21.15.33:0/19431397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112f7fe640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11180040e0 0x7f11301f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112ffff640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f113015fd40 0x7f11300c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 -- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f113015fd40 msgr2=0x7f11300c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f113015fd40 0x7f11300c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 -- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11180040e0 msgr2=0x7f11301f3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11180040e0 0x7f11301f3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 -- 172.21.15.33:0/19431397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11300cbf50 con 0x7f11301511a0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f112effd640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11301511a0 0x7f113015f800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11240b1040 tx=0x7f1118005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f1117fff640 1 -- 172.21.15.33:0/19431397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240bc070 con 0x7f11301511a0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f1117fff640 1 -- 172.21.15.33:0/19431397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11240af610 con 0x7f11301511a0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11300cc1f0 con 0x7f11301511a0 2024-04-25T01:08:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f1117fff640 1 -- 172.21.15.33:0/19431397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11240ab040 con 0x7f11301511a0 2024-04-25T01:08:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11300cc7b0 con 0x7f11301511a0 2024-04-25T01:08:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f1117fff640 1 -- 172.21.15.33:0/19431397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f11240bf070 con 0x7f11301511a0 2024-04-25T01:08:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f1117fff640 1 -- 172.21.15.33:0/19431397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11240c1710 con 0x7f11301511a0 2024-04-25T01:08:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.474+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1130069040 con 0x7f11301511a0 2024-04-25T01:08:06.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.498+0000 7f1117fff640 1 -- 172.21.15.33:0/19431397 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f11240d8dd0 con 0x7f11301511a0 2024-04-25T01:08:06.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.558+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f11300b6790 con 0x7f11301511a0 2024-04-25T01:08:06.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f1117fff640 1 -- 172.21.15.33:0/19431397 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f11240c7020 con 0x7f11301511a0 2024-04-25T01:08:06.558 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-04-25T01:08:06.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11301511a0 msgr2=0x7f113015f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:06.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f113612e640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11301511a0 0x7f113015f800 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f11240b1040 tx=0x7f1118005840 comp rx=0 tx=0).stop 2024-04-25T01:08:06.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 shutdown_connections 2024-04-25T01:08:06.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f113612e640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f113015fd40 0x7f11300c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f113612e640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11301511a0 0x7f113015f800 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f113612e640 1 --2- 172.21.15.33:0/19431397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11180040e0 0x7f11301f3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:06.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.562+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 >> 172.21.15.33:0/19431397 conn(0x7f11300d3dd0 msgr2=0x7f11300d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:06.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.566+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 shutdown_connections 2024-04-25T01:08:06.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:06.566+0000 7f113612e640 1 -- 172.21.15.33:0/19431397 wait complete. 2024-04-25T01:08:06.579 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T01:08:06.579 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T01:08:06.579 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:06.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T01:08:06.590 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:06.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T01:08:06.647 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:06.647 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T01:08:06.650 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:06.658+0000 7f694bd44180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:08:06.673 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:06.678+0000 7fa91274d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T01:08:06.684 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:06.684 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T01:08:06.743 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:06.743 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T01:08:06.756 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:06.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T01:08:06.810 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:06.810 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T01:08:06.865 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:08:06.865 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T01:08:06.917 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new a55c2d3b-5b10-4106-a9b3-2041d0435aa0 0 2024-04-25T01:08:07.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f7947332640 1 Processor -- start 2024-04-25T01:08:07.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f7947332640 1 -- start start 2024-04-25T01:08:07.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f7947332640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f794015c040 0x7f7940072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f7947332640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79400af620 0x7f7940072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f7947332640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79400b2a20 con 0x7f7940073390 2024-04-25T01:08:07.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f7947332640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f79400b0150 con 0x7f794015c040 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f7947332640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f79400b0430 con 0x7f79400af620 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f79448cd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79400af620 0x7f7940072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f79448cd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79400af620 0x7f7940072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59914/0 (socket says 172.21.15.138:59914) 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.090+0000 7f79448cd640 1 -- 172.21.15.138:0/9303499 learned_addr learned my addr 172.21.15.138:0/9303499 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f79450ce640 1 --2- 172.21.15.138:0/9303499 >> v2:172.21.15.33:3300/0 conn(0x7f794015c040 0x7f7940072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f79448cd640 1 -- 172.21.15.138:0/9303499 >> v2:172.21.15.33:3300/0 conn(0x7f794015c040 msgr2=0x7f7940072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f79448cd640 1 --2- 172.21.15.138:0/9303499 >> v2:172.21.15.33:3300/0 conn(0x7f794015c040 0x7f7940072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f79448cd640 1 -- 172.21.15.138:0/9303499 >> v1:172.21.15.33:6789/0 conn(0x7f7940073390 legacy=0x7f79400b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f79448cd640 1 -- 172.21.15.138:0/9303499 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79400b0710 con 0x7f79400af620 2024-04-25T01:08:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f79450ce640 1 --2- 172.21.15.138:0/9303499 >> v2:172.21.15.33:3300/0 conn(0x7f794015c040 0x7f7940072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:07.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f79448cd640 1 --2- 172.21.15.138:0/9303499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f7940072e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79400c3d90 tx=0x7f793001c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82c05d643b677c91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7937fff640 1 -- 172.21.15.138:0/9303499 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300b0070 con 0x7f79400af620 2024-04-25T01:08:07.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7937fff640 1 -- 172.21.15.138:0/9303499 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79300ab600 con 0x7f79400af620 2024-04-25T01:08:07.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7937fff640 1 -- 172.21.15.138:0/9303499 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79300ab9d0 con 0x7f79400af620 2024-04-25T01:08:07.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7947332640 1 -- 172.21.15.138:0/9303499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 msgr2=0x7f7940072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7947332640 1 --2- 172.21.15.138:0/9303499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f7940072e50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f79400c3d90 tx=0x7f793001c520 comp rx=0 tx=0).stop 2024-04-25T01:08:07.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7947332640 1 -- 172.21.15.138:0/9303499 shutdown_connections 2024-04-25T01:08:07.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7947332640 1 --2- 172.21.15.138:0/9303499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f7940072e50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7947332640 1 --2- 172.21.15.138:0/9303499 >> v2:172.21.15.33:3300/0 conn(0x7f794015c040 0x7f7940072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.094+0000 7f7947332640 1 -- 172.21.15.138:0/9303499 >> 172.21.15.138:0/9303499 conn(0x7f79400b1e60 msgr2=0x7f79400b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- 172.21.15.138:0/9303499 shutdown_connections 2024-04-25T01:08:07.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- 172.21.15.138:0/9303499 wait complete. 2024-04-25T01:08:07.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 Processor -- start 2024-04-25T01:08:07.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- start start 2024-04-25T01:08:07.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7940073390 0x7f79400c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f79400ba860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f794015c040 0x7f79400bada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7940168be0 con 0x7f7940073390 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7940168a60 con 0x7f79400af620 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7940168d60 con 0x7f794015c040 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f794015c040 0x7f79400bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79448cd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f79400ba860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79448cd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f79400ba860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59918/0 (socket says 172.21.15.138:59918) 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f794015c040 0x7f79400bada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53702/0 (socket says 172.21.15.138:53702) 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79448cd640 1 -- 172.21.15.138:0/3969034098 learned_addr learned my addr 172.21.15.138:0/3969034098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79450ce640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7940073390 0x7f79400c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 -- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 msgr2=0x7f79400ba860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f79400ba860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 -- 172.21.15.138:0/3969034098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7940073390 msgr2=0x7f79400c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7940073390 0x7f79400c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 -- 172.21.15.138:0/3969034098 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79401570d0 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79448cd640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f79400ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f79458cf640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f794015c040 0x7f79400bada0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c390 tx=0x7f793c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7935ffb640 1 -- 172.21.15.138:0/3969034098 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c017070 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7935ffb640 1 -- 172.21.15.138:0/3969034098 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793c0052d0 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79400ce000 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7935ffb640 1 -- 172.21.15.138:0/3969034098 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c003840 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f79400ce5c0 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7935ffb640 1 -- 172.21.15.138:0/3969034098 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f793c01a070 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7935ffb640 1 -- 172.21.15.138:0/3969034098 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f793c01c440 con 0x7f794015c040 2024-04-25T01:08:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.098+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7940069040 con 0x7f794015c040 2024-04-25T01:08:07.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.114+0000 7f7935ffb640 1 -- 172.21.15.138:0/3969034098 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f793c021020 con 0x7f794015c040 2024-04-25T01:08:07.126 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:07.134+0000 7f694bd44180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:08:07.138 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:07.142+0000 7fa91274d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T01:08:07.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.182+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a55c2d3b-5b10-4106-a9b3-2041d0435aa0", "id": 0} v 0) -- 0x7f79400cdd30 con 0x7f794015c040 2024-04-25T01:08:07.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.186+0000 7f7935ffb640 1 -- 172.21.15.138:0/3969034098 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a55c2d3b-5b10-4106-a9b3-2041d0435aa0", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f793c011590 con 0x7f794015c040 2024-04-25T01:08:07.181 INFO:teuthology.orchestra.run.smithi138.stdout:0 2024-04-25T01:08:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f794015c040 msgr2=0x7f79400bada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f794015c040 0x7f79400bada0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c390 tx=0x7f793c00c850 comp rx=0 tx=0).stop 2024-04-25T01:08:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 shutdown_connections 2024-04-25T01:08:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f794015c040 0x7f79400bada0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79400af620 0x7f79400ba860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 --2- 172.21.15.138:0/3969034098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7940073390 0x7f79400c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 >> 172.21.15.138:0/3969034098 conn(0x7f79400b1e60 msgr2=0x7f794015b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 shutdown_connections 2024-04-25T01:08:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.190+0000 7f7947332640 1 -- 172.21.15.138:0/3969034098 wait complete. 2024-04-25T01:08:07.198 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new b941572f-acd0-4d94-af10-64f4735614a5 1 2024-04-25T01:08:07.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f8860259640 1 Processor -- start 2024-04-25T01:08:07.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f8860259640 1 -- start start 2024-04-25T01:08:07.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f8860259640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580cd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f8860259640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f885806f950 0x7f88580cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f8860259640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88580cdb80 con 0x7f88580cf6c0 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f8860259640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f88580d0850 con 0x7f885806f950 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f8860259640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f88580cfc40 con 0x7f88580cd260 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885dff5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885dff5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53706/0 (socket says 172.21.15.138:53706) 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885dff5640 1 -- 172.21.15.138:0/3935442338 learned_addr learned my addr 172.21.15.138:0/3935442338 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885d7f4640 1 --2- 172.21.15.138:0/3935442338 >> v2:172.21.15.33:3300/0 conn(0x7f885806f950 0x7f88580cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885dff5640 1 -- 172.21.15.138:0/3935442338 >> v2:172.21.15.33:3300/0 conn(0x7f885806f950 msgr2=0x7f88580cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885dff5640 1 --2- 172.21.15.138:0/3935442338 >> v2:172.21.15.33:3300/0 conn(0x7f885806f950 0x7f88580cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885dff5640 1 -- 172.21.15.138:0/3935442338 >> v1:172.21.15.33:6789/0 conn(0x7f88580cf6c0 legacy=0x7f8858071d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:07.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.318+0000 7f885dff5640 1 -- 172.21.15.138:0/3935442338 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88580cff20 con 0x7f88580cd260 2024-04-25T01:08:07.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885dff5640 1 --2- 172.21.15.138:0/3935442338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580cd640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8854009870 tx=0x7f885401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b54b39ebe66a6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885cff3640 1 -- 172.21.15.138:0/3935442338 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540aa610 con 0x7f88580cd260 2024-04-25T01:08:07.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885cff3640 1 -- 172.21.15.138:0/3935442338 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88540af070 con 0x7f88580cd260 2024-04-25T01:08:07.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885cff3640 1 -- 172.21.15.138:0/3935442338 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88540aa9d0 con 0x7f88580cd260 2024-04-25T01:08:07.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- 172.21.15.138:0/3935442338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 msgr2=0x7f88580cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 --2- 172.21.15.138:0/3935442338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580cd640 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8854009870 tx=0x7f885401c4f0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- 172.21.15.138:0/3935442338 shutdown_connections 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 --2- 172.21.15.138:0/3935442338 >> v2:172.21.15.33:3300/0 conn(0x7f885806f950 0x7f88580cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 --2- 172.21.15.138:0/3935442338 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580cd640 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- 172.21.15.138:0/3935442338 >> 172.21.15.138:0/3935442338 conn(0x7f88580b2820 msgr2=0x7f88580b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- 172.21.15.138:0/3935442338 shutdown_connections 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- 172.21.15.138:0/3935442338 wait complete. 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 Processor -- start 2024-04-25T01:08:07.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- start start 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f885806f950 0x7f88580afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88580cf6c0 0x7f885812ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f88581687a0 con 0x7f885806f950 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8858168620 con 0x7f88580cf6c0 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f8860259640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8858168920 con 0x7f88580cd260 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885e7f6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88580cf6c0 0x7f885812ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53708/0 (socket says 172.21.15.138:53708) 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 -- 172.21.15.138:0/601465850 learned_addr learned my addr 172.21.15.138:0/601465850 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885e7f6640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88580cf6c0 0x7f885812ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59932/0 (socket says 172.21.15.138:59932) 2024-04-25T01:08:07.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885dff5640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f885806f950 0x7f88580afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 -- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88580cf6c0 msgr2=0x7f885812ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88580cf6c0 0x7f885812ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 -- 172.21.15.138:0/601465850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f885806f950 msgr2=0x7f88580afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f885806f950 0x7f88580afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 -- 172.21.15.138:0/601465850 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8858159000 con 0x7f88580cd260 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885e7f6640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88580cf6c0 0x7f885812ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.322+0000 7f885d7f4640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580afff0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f884800c7f0 tx=0x7f884800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f884e7fc640 1 -- 172.21.15.138:0/601465850 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848015070 con 0x7f88580cd260 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f884e7fc640 1 -- 172.21.15.138:0/601465850 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8848007820 con 0x7f88580cd260 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f884e7fc640 1 -- 172.21.15.138:0/601465850 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8848007c10 con 0x7f88580cd260 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88581592e0 con 0x7f88580cd260 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8858159b80 con 0x7f88580cd260 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f884e7fc640 1 -- 172.21.15.138:0/601465850 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8848018070 con 0x7f88580cd260 2024-04-25T01:08:07.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f884e7fc640 1 -- 172.21.15.138:0/601465850 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f8848011a00 con 0x7f88580cd260 2024-04-25T01:08:07.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88581596d0 con 0x7f88580cd260 2024-04-25T01:08:07.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.326+0000 7f884e7fc640 1 -- 172.21.15.138:0/601465850 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8848011cb0 con 0x7f88580cd260 2024-04-25T01:08:07.329 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:07.334+0000 7f694bd44180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:08:07.337 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:07.342+0000 7fa91274d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T01:08:07.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.462+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b941572f-acd0-4d94-af10-64f4735614a5", "id": 1} v 0) -- 0x7f88580d1980 con 0x7f88580cd260 2024-04-25T01:08:07.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.470+0000 7f884e7fc640 1 -- 172.21.15.138:0/601465850 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b941572f-acd0-4d94-af10-64f4735614a5", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f884801a4a0 con 0x7f88580cd260 2024-04-25T01:08:07.466 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 msgr2=0x7f88580afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580afff0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f884800c7f0 tx=0x7f884800ccb0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 shutdown_connections 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88580cf6c0 0x7f885812ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f88580cd260 0x7f88580afff0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 --2- 172.21.15.138:0/601465850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f885806f950 0x7f88580afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 >> 172.21.15.138:0/601465850 conn(0x7f88580b2820 msgr2=0x7f88580d0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 shutdown_connections 2024-04-25T01:08:07.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.474+0000 7f8860259640 1 -- 172.21.15.138:0/601465850 wait complete. 2024-04-25T01:08:07.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 36aa7662-65b1-413e-a94f-099070af8a2d 2 2024-04-25T01:08:07.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdd6db3640 1 Processor -- start 2024-04-25T01:08:07.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdd6db3640 1 -- start start 2024-04-25T01:08:07.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdd6db3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d2310 0x7fcdd006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdd6db3640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd0070860 con 0x7fcdd0156600 2024-04-25T01:08:07.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdd6db3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd01569e0 con 0x7fcdd00d0b80 2024-04-25T01:08:07.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdd6db3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd006f440 con 0x7fcdd00d2310 2024-04-25T01:08:07.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdcffff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcdd00d0b80 0x7fcdd0163b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42108/0 (socket says 172.21.15.138:42108) 2024-04-25T01:08:07.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdcffff640 1 -- 172.21.15.138:0/4001158981 learned_addr learned my addr 172.21.15.138:0/4001158981 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T01:08:07.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.610+0000 7fcdd5350640 1 --2- 172.21.15.138:0/4001158981 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d2310 0x7fcdd006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd5350640 1 -- 172.21.15.138:0/4001158981 >> v1:172.21.15.138:6789/0 conn(0x7fcdd00d0b80 legacy=0x7fcdd0163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdcf7fe640 1 -- 172.21.15.138:0/4001158981 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418665995 0 0) 0x7fcdd01569e0 con 0x7fcdd00d0b80 2024-04-25T01:08:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd5350640 1 -- 172.21.15.138:0/4001158981 >> v1:172.21.15.33:6789/0 conn(0x7fcdd0156600 legacy=0x7fcdd00d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd5350640 1 -- 172.21.15.138:0/4001158981 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd015d470 con 0x7fcdd00d2310 2024-04-25T01:08:07.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd5350640 1 --2- 172.21.15.138:0/4001158981 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d2310 0x7fcdd006ef00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009dc0 tx=0x7fcdc801c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921b61d7befe31da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdcf7fe640 1 -- 172.21.15.138:0/4001158981 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80b2070 con 0x7fcdd00d2310 2024-04-25T01:08:07.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdcf7fe640 1 -- 172.21.15.138:0/4001158981 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc80a57c0 con 0x7fcdd00d2310 2024-04-25T01:08:07.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdcf7fe640 1 -- 172.21.15.138:0/4001158981 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80a5b90 con 0x7fcdd00d2310 2024-04-25T01:08:07.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd6db3640 1 -- 172.21.15.138:0/4001158981 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d2310 msgr2=0x7fcdd006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd6db3640 1 --2- 172.21.15.138:0/4001158981 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d2310 0x7fcdd006ef00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009dc0 tx=0x7fcdc801c900 comp rx=0 tx=0).stop 2024-04-25T01:08:07.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd6db3640 1 -- 172.21.15.138:0/4001158981 shutdown_connections 2024-04-25T01:08:07.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd6db3640 1 --2- 172.21.15.138:0/4001158981 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d2310 0x7fcdd006ef00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd6db3640 1 -- 172.21.15.138:0/4001158981 >> 172.21.15.138:0/4001158981 conn(0x7fcdd009a9d0 msgr2=0x7fcdd009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd6db3640 1 -- 172.21.15.138:0/4001158981 shutdown_connections 2024-04-25T01:08:07.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.614+0000 7fcdd6db3640 1 -- 172.21.15.138:0/4001158981 wait complete. 2024-04-25T01:08:07.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 Processor -- start 2024-04-25T01:08:07.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 -- start start 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d0b80 0x7fcdd0137140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00d2310 0x7fcdd014d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd0156600 0x7fcdd01388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd015d130 con 0x7fcdd00d2310 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd015cfb0 con 0x7fcdd0156600 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd0169ac0 con 0x7fcdd00d0b80 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d0b80 0x7fcdd0137140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d0b80 0x7fcdd0137140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53718/0 (socket says 172.21.15.138:53718) 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 -- 172.21.15.138:0/1412992697 learned_addr learned my addr 172.21.15.138:0/1412992697 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd5350640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd0156600 0x7fcdd01388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdcffff640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00d2310 0x7fcdd014d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 -- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd0156600 msgr2=0x7fcdd01388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd0156600 0x7fcdd01388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 -- 172.21.15.138:0/1412992697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00d2310 msgr2=0x7fcdd014d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00d2310 0x7fcdd014d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 -- 172.21.15.138:0/1412992697 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd0137800 con 0x7fcdd00d0b80 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd4b4f640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d0b80 0x7fcdd0137140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c3d0 tx=0x7fcdc000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdcd7fa640 1 -- 172.21.15.138:0/1412992697 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0017070 con 0x7fcdd00d0b80 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdcd7fa640 1 -- 172.21.15.138:0/1412992697 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdc00052f0 con 0x7fcdd00d0b80 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd0137ae0 con 0x7fcdd00d0b80 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdcd7fa640 1 -- 172.21.15.138:0/1412992697 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0003860 con 0x7fcdd00d0b80 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd01380a0 con 0x7fcdd00d0b80 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdcd7fa640 1 -- 172.21.15.138:0/1412992697 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcdc001a070 con 0x7fcdd00d0b80 2024-04-25T01:08:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdcd7fa640 1 -- 172.21.15.138:0/1412992697 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fcdc0012420 con 0x7fcdd00d0b80 2024-04-25T01:08:07.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.618+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd0069040 con 0x7fcdd00d0b80 2024-04-25T01:08:07.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.622+0000 7fcdcd7fa640 1 -- 172.21.15.138:0/1412992697 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcdc0021070 con 0x7fcdd00d0b80 2024-04-25T01:08:07.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.690+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "36aa7662-65b1-413e-a94f-099070af8a2d", "id": 2} v 0) -- 0x7fcdd014e8e0 con 0x7fcdd00d0b80 2024-04-25T01:08:07.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.694+0000 7fcdcd7fa640 1 -- 172.21.15.138:0/1412992697 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "36aa7662-65b1-413e-a94f-099070af8a2d", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fcdc0011650 con 0x7fcdd00d0b80 2024-04-25T01:08:07.689 INFO:teuthology.orchestra.run.smithi138.stdout:2 2024-04-25T01:08:07.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.694+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d0b80 msgr2=0x7fcdd0137140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d0b80 0x7fcdd0137140 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcdc000c3d0 tx=0x7fcdc000c890 comp rx=0 tx=0).stop 2024-04-25T01:08:07.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 shutdown_connections 2024-04-25T01:08:07.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd0156600 0x7fcdd01388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd00d2310 0x7fcdd014d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 --2- 172.21.15.138:0/1412992697 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcdd00d0b80 0x7fcdd0137140 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 >> 172.21.15.138:0/1412992697 conn(0x7fcdd009a9d0 msgr2=0x7fcdd00b6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 shutdown_connections 2024-04-25T01:08:07.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.698+0000 7fcdd6db3640 1 -- 172.21.15.138:0/1412992697 wait complete. 2024-04-25T01:08:07.710 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 3d561dd1-ed88-488d-87b6-f3017b4dc353 3 2024-04-25T01:08:07.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.834+0000 7f1fc4545640 1 Processor -- start 2024-04-25T01:08:07.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.834+0000 7f1fc4545640 1 -- start start 2024-04-25T01:08:07.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fbc0b4570 con 0x7f1fbc0b3ff0 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fbc0b4850 con 0x7f1fbc0cd480 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc0cf190 con 0x7f1fbc15c360 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc1ae0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc22e1640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f1fbc0cd480 0x7f1fbc0cd860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42120/0 (socket says 172.21.15.138:42120) 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc22e1640 1 -- 172.21.15.138:0/1867423161 learned_addr learned my addr 172.21.15.138:0/1867423161 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc12df640 1 -- 172.21.15.138:0/1867423161 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1147754657 0 0) 0x7f1fbc0b4850 con 0x7f1fbc0cd480 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc12df640 1 -- 172.21.15.138:0/1867423161 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa8005740 con 0x7f1fbc0cd480 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc1ae0640 1 -- 172.21.15.138:0/1867423161 >> v1:172.21.15.138:6789/0 conn(0x7f1fbc0cd480 legacy=0x7f1fbc0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc1ae0640 1 -- 172.21.15.138:0/1867423161 >> v1:172.21.15.33:6789/0 conn(0x7f1fbc0b3ff0 legacy=0x7f1fbc0cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:07.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc1ae0640 1 -- 172.21.15.138:0/1867423161 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc0b4b30 con 0x7f1fbc15c360 2024-04-25T01:08:07.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc1ae0640 1 --2- 172.21.15.138:0/1867423161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc0b3ab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097f0 tx=0x7f1fac01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed966fd6a8badd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc12df640 1 -- 172.21.15.138:0/1867423161 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0b0070 con 0x7f1fbc15c360 2024-04-25T01:08:07.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc12df640 1 -- 172.21.15.138:0/1867423161 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac0ab600 con 0x7f1fbc15c360 2024-04-25T01:08:07.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc12df640 1 -- 172.21.15.138:0/1867423161 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0ab9d0 con 0x7f1fbc15c360 2024-04-25T01:08:07.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- 172.21.15.138:0/1867423161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 msgr2=0x7f1fbc0b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 --2- 172.21.15.138:0/1867423161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc0b3ab0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fac0097f0 tx=0x7f1fac01c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- 172.21.15.138:0/1867423161 shutdown_connections 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 --2- 172.21.15.138:0/1867423161 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc0b3ab0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- 172.21.15.138:0/1867423161 >> 172.21.15.138:0/1867423161 conn(0x7f1fbc0b27b0 msgr2=0x7f1fbc0b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- 172.21.15.138:0/1867423161 shutdown_connections 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.838+0000 7f1fc4545640 1 -- 172.21.15.138:0/1867423161 wait complete. 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 Processor -- start 2024-04-25T01:08:07.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 -- start start 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fbc0b3ff0 0x7f1fbc06ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc0cd480 0x7f1fbc15a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc06b660 con 0x7f1fbc0b3ff0 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1fbc06b4e0 con 0x7f1fbc0cd480 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1fbc06b7e0 con 0x7f1fbc15c360 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc1ae0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc0cd480 0x7f1fbc15a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc142cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53736/0 (socket says 172.21.15.138:53736) 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 -- 172.21.15.138:0/3515473346 learned_addr learned my addr 172.21.15.138:0/3515473346 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc22e1640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fbc0b3ff0 0x7f1fbc06ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 -- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc0cd480 msgr2=0x7f1fbc15a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc0cd480 0x7f1fbc15a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 -- 172.21.15.138:0/3515473346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fbc0b3ff0 msgr2=0x7f1fbc06ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fbc0b3ff0 0x7f1fbc06ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 -- 172.21.15.138:0/3515473346 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fbc158de0 con 0x7f1fbc15c360 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc1ae0640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc0cd480 0x7f1fbc15a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc2ae2640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc142cc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c780 tx=0x7f1fb400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fb2ffd640 1 -- 172.21.15.138:0/3515473346 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4004b70 con 0x7f1fbc15c360 2024-04-25T01:08:07.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fb2ffd640 1 -- 172.21.15.138:0/3515473346 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb4004d10 con 0x7f1fbc15c360 2024-04-25T01:08:07.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fb2ffd640 1 -- 172.21.15.138:0/3515473346 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400aa90 con 0x7f1fbc15c360 2024-04-25T01:08:07.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fbc1590c0 con 0x7f1fbc15c360 2024-04-25T01:08:07.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fbc159680 con 0x7f1fbc15c360 2024-04-25T01:08:07.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fb2ffd640 1 -- 172.21.15.138:0/3515473346 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1fb4006b60 con 0x7f1fbc15c360 2024-04-25T01:08:07.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fb2ffd640 1 -- 172.21.15.138:0/3515473346 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1fb4007640 con 0x7f1fbc15c360 2024-04-25T01:08:07.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.842+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fbc069040 con 0x7f1fbc15c360 2024-04-25T01:08:07.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.846+0000 7f1fb2ffd640 1 -- 172.21.15.138:0/3515473346 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1fb4020070 con 0x7f1fbc15c360 2024-04-25T01:08:07.905 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:07.910+0000 7fa91274d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:08:07.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.914+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3d561dd1-ed88-488d-87b6-f3017b4dc353", "id": 3} v 0) -- 0x7f1fbc0cca90 con 0x7f1fbc15c360 2024-04-25T01:08:07.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.918+0000 7f1fb2ffd640 1 -- 172.21.15.138:0/3515473346 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3d561dd1-ed88-488d-87b6-f3017b4dc353", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1fb40071b0 con 0x7f1fbc15c360 2024-04-25T01:08:07.914 INFO:teuthology.orchestra.run.smithi138.stdout:3 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 msgr2=0x7f1fbc142cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc142cc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c780 tx=0x7f1fb400cc40 comp rx=0 tx=0).stop 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 shutdown_connections 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1fbc15c360 0x7f1fbc142cc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fbc0cd480 0x7f1fbc15a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 --2- 172.21.15.138:0/3515473346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fbc0b3ff0 0x7f1fbc06ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 >> 172.21.15.138:0/3515473346 conn(0x7f1fbc0b27b0 msgr2=0x7f1fbc0747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 shutdown_connections 2024-04-25T01:08:07.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:07.922+0000 7f1fc4545640 1 -- 172.21.15.138:0/3515473346 wait complete. 2024-04-25T01:08:07.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new ff949513-c8e5-489b-8992-5b673f1e1cc2 4 2024-04-25T01:08:07.960 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:07.966+0000 7fa91274d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:08:07.971 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:07.978+0000 7f694bd44180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T01:08:08.014 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:08.022+0000 7fa91274d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:08:08.027 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.034+0000 7f694bd44180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T01:08:08.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.050+0000 7f8be9888640 1 Processor -- start 2024-04-25T01:08:08.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.050+0000 7f8be9888640 1 -- start start 2024-04-25T01:08:08.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be9888640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be406c930 0x7f8be40ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be9888640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8be40cfae0 0x7f8be406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be9888640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be40cc840 con 0x7f8be415be70 2024-04-25T01:08:08.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be9888640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8be40ccb20 con 0x7f8be40cfae0 2024-04-25T01:08:08.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be9888640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8be40cce00 con 0x7f8be406c930 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be406c930 0x7f8be40ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be406c930 0x7f8be40ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53750/0 (socket says 172.21.15.138:53750) 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 -- 172.21.15.138:0/1175828994 learned_addr learned my addr 172.21.15.138:0/1175828994 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be37fe640 1 --2- 172.21.15.138:0/1175828994 >> v2:172.21.15.33:3300/0 conn(0x7f8be40cfae0 0x7f8be406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 -- 172.21.15.138:0/1175828994 >> v2:172.21.15.33:3300/0 conn(0x7f8be40cfae0 msgr2=0x7f8be406d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 --2- 172.21.15.138:0/1175828994 >> v2:172.21.15.33:3300/0 conn(0x7f8be40cfae0 0x7f8be406d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 -- 172.21.15.138:0/1175828994 >> v1:172.21.15.33:6789/0 conn(0x7f8be415be70 legacy=0x7f8be406c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:08.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 -- 172.21.15.138:0/1175828994 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be40cd0e0 con 0x7f8be406c930 2024-04-25T01:08:08.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be27fc640 1 --2- 172.21.15.138:0/1175828994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be406c930 0x7f8be40ce300 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=151e27df891f3c6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be1ffb640 1 -- 172.21.15.138:0/1175828994 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80b0070 con 0x7f8be406c930 2024-04-25T01:08:08.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be1ffb640 1 -- 172.21.15.138:0/1175828994 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd80ab600 con 0x7f8be406c930 2024-04-25T01:08:08.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.054+0000 7f8be1ffb640 1 -- 172.21.15.138:0/1175828994 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd80ab9d0 con 0x7f8be406c930 2024-04-25T01:08:08.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- 172.21.15.138:0/1175828994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be406c930 msgr2=0x7f8be40ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 --2- 172.21.15.138:0/1175828994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be406c930 0x7f8be40ce300 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd801c520 comp rx=0 tx=0).stop 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- 172.21.15.138:0/1175828994 shutdown_connections 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 --2- 172.21.15.138:0/1175828994 >> v2:172.21.15.33:3300/0 conn(0x7f8be40cfae0 0x7f8be406d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 --2- 172.21.15.138:0/1175828994 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be406c930 0x7f8be40ce300 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- 172.21.15.138:0/1175828994 >> 172.21.15.138:0/1175828994 conn(0x7f8be40715d0 msgr2=0x7f8be40719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- 172.21.15.138:0/1175828994 shutdown_connections 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- 172.21.15.138:0/1175828994 wait complete. 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 Processor -- start 2024-04-25T01:08:08.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- start start 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be406c930 0x7f8be4144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be40cfae0 0x7f8be415c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be415be70 0x7f8be415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8be406e900 con 0x7f8be406c930 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8be406e780 con 0x7f8be40cfae0 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8be406ea80 con 0x7f8be415be70 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be415be70 0x7f8be415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be27fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be40cfae0 0x7f8be415c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be415be70 0x7f8be415cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53756/0 (socket says 172.21.15.138:53756) 2024-04-25T01:08:08.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 -- 172.21.15.138:0/4029830414 learned_addr learned my addr 172.21.15.138:0/4029830414 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be2ffd640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be406c930 0x7f8be4144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 -- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be40cfae0 msgr2=0x7f8be415c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be40cfae0 0x7f8be415c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 -- 172.21.15.138:0/4029830414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be406c930 msgr2=0x7f8be4144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be406c930 0x7f8be4144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 -- 172.21.15.138:0/4029830414 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be406c760 con 0x7f8be415be70 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be2ffd640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be406c930 0x7f8be4144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be27fc640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be40cfae0 0x7f8be415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be37fe640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be415be70 0x7f8be415cf00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc007e60 tx=0x7f8bdc00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8bcbfff640 1 -- 172.21.15.138:0/4029830414 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc014420 con 0x7f8be415be70 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8bcbfff640 1 -- 172.21.15.138:0/4029830414 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bdc019070 con 0x7f8be415be70 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be415dd90 con 0x7f8be415be70 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8bcbfff640 1 -- 172.21.15.138:0/4029830414 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc014820 con 0x7f8be415be70 2024-04-25T01:08:08.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be415dab0 con 0x7f8be415be70 2024-04-25T01:08:08.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8bcbfff640 1 -- 172.21.15.138:0/4029830414 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8bdc0149c0 con 0x7f8be415be70 2024-04-25T01:08:08.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.058+0000 7f8bcbfff640 1 -- 172.21.15.138:0/4029830414 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f8bdc01d8d0 con 0x7f8be415be70 2024-04-25T01:08:08.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.062+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be4069040 con 0x7f8be415be70 2024-04-25T01:08:08.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.062+0000 7f8bcbfff640 1 -- 172.21.15.138:0/4029830414 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8bdc01d3c0 con 0x7f8be415be70 2024-04-25T01:08:08.082 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.090+0000 7f694bd44180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T01:08:08.095 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:08.102+0000 7fa91274d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:08:08.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.130+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ff949513-c8e5-489b-8992-5b673f1e1cc2", "id": 4} v 0) -- 0x7f8be406c500 con 0x7f8be415be70 2024-04-25T01:08:08.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8bcbfff640 1 -- 172.21.15.138:0/4029830414 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ff949513-c8e5-489b-8992-5b673f1e1cc2", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8bdc01d5a0 con 0x7f8be415be70 2024-04-25T01:08:08.130 INFO:teuthology.orchestra.run.smithi138.stdout:4 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be415be70 msgr2=0x7f8be415cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be415be70 0x7f8be415cf00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc007e60 tx=0x7f8bdc00e4b0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 shutdown_connections 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8be415be70 0x7f8be415cf00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8be40cfae0 0x7f8be415c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 --2- 172.21.15.138:0/4029830414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8be406c930 0x7f8be4144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 >> 172.21.15.138:0/4029830414 conn(0x7f8be40715d0 msgr2=0x7f8be40b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 shutdown_connections 2024-04-25T01:08:08.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.138+0000 7f8be9888640 1 -- 172.21.15.138:0/4029830414 wait complete. 2024-04-25T01:08:08.147 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 2512be72-afd8-4caf-b076-9c0a07dbac06 5 2024-04-25T01:08:08.150 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:08.158+0000 7fa91274d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:08:08.162 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.170+0000 7f694bd44180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T01:08:08.216 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.222+0000 7f694bd44180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T01:08:08.231 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:08.238+0000 7fa91274d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:08:08.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853f65e640 1 Processor -- start 2024-04-25T01:08:08.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853f65e640 1 -- start start 2024-04-25T01:08:08.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853f65e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b5df0 0x7f853806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853f65e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85380b4230 0x7f853806e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853f65e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8538161f20 con 0x7f8538147410 2024-04-25T01:08:08.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853f65e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8538162200 con 0x7f85380b4230 2024-04-25T01:08:08.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853f65e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85381624e0 con 0x7f85380b5df0 2024-04-25T01:08:08.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853cbf9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b5df0 0x7f853806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853d3fa640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8538147410 0x7f85380b3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42136/0 (socket says 172.21.15.138:42136) 2024-04-25T01:08:08.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853dbfb640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85380b4230 0x7f853806e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853d3fa640 1 -- 172.21.15.138:0/1687621088 learned_addr learned my addr 172.21.15.138:0/1687621088 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T01:08:08.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.278+0000 7f853cbf9640 1 -- 172.21.15.138:0/1687621088 >> v2:172.21.15.138:3300/0 conn(0x7f85380b4230 msgr2=0x7f853806e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:08.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853cbf9640 1 --2- 172.21.15.138:0/1687621088 >> v2:172.21.15.138:3300/0 conn(0x7f85380b4230 0x7f853806e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853cbf9640 1 -- 172.21.15.138:0/1687621088 >> v1:172.21.15.138:6789/0 conn(0x7f8538147410 legacy=0x7f85380b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f852ffff640 1 -- 172.21.15.138:0/1687621088 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4038375521 0 0) 0x7f8538161f20 con 0x7f8538147410 2024-04-25T01:08:08.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853cbf9640 1 -- 172.21.15.138:0/1687621088 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85381627c0 con 0x7f85380b5df0 2024-04-25T01:08:08.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853dbfb640 1 --2- 172.21.15.138:0/1687621088 >> v2:172.21.15.138:3300/0 conn(0x7f85380b4230 0x7f853806e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:08.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853cbf9640 1 --2- 172.21.15.138:0/1687621088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b5df0 0x7f853806bc20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f85280097c0 tx=0x7f852801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8a98c6ea3f56b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f852ffff640 1 -- 172.21.15.138:0/1687621088 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280b0070 con 0x7f85380b5df0 2024-04-25T01:08:08.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f852ffff640 1 -- 172.21.15.138:0/1687621088 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85280ab600 con 0x7f85380b5df0 2024-04-25T01:08:08.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f852ffff640 1 -- 172.21.15.138:0/1687621088 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85280ab9d0 con 0x7f85380b5df0 2024-04-25T01:08:08.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 -- 172.21.15.138:0/1687621088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b5df0 msgr2=0x7f853806bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 --2- 172.21.15.138:0/1687621088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b5df0 0x7f853806bc20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f85280097c0 tx=0x7f852801c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 -- 172.21.15.138:0/1687621088 shutdown_connections 2024-04-25T01:08:08.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 --2- 172.21.15.138:0/1687621088 >> v2:172.21.15.138:3300/0 conn(0x7f85380b4230 0x7f853806e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 --2- 172.21.15.138:0/1687621088 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b5df0 0x7f853806bc20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 -- 172.21.15.138:0/1687621088 >> 172.21.15.138:0/1687621088 conn(0x7f85380b2820 msgr2=0x7f85380b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 -- 172.21.15.138:0/1687621088 shutdown_connections 2024-04-25T01:08:08.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.282+0000 7f853f65e640 1 -- 172.21.15.138:0/1687621088 wait complete. 2024-04-25T01:08:08.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 Processor -- start 2024-04-25T01:08:08.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 -- start start 2024-04-25T01:08:08.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b4230 0x7f853815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85380b5df0 0x7f853815f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8538147410 0x7f8538146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f853806f250 con 0x7f85380b5df0 2024-04-25T01:08:08.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f853806f0d0 con 0x7f8538147410 2024-04-25T01:08:08.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f853806f3d0 con 0x7f85380b4230 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8538147410 0x7f8538146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853d3fa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b4230 0x7f853815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8538147410 0x7f8538146720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59978/0 (socket says 172.21.15.138:59978) 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 -- 172.21.15.138:0/2129959570 learned_addr learned my addr 172.21.15.138:0/2129959570 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853d3fa640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b4230 0x7f853815ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53776/0 (socket says 172.21.15.138:53776) 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 -- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b4230 msgr2=0x7f853815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b4230 0x7f853815ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 -- 172.21.15.138:0/2129959570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85380b5df0 msgr2=0x7f853815f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85380b5df0 0x7f853815f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 -- 172.21.15.138:0/2129959570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8538146c60 con 0x7f8538147410 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853d3fa640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b4230 0x7f853815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853dbfb640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8538147410 0x7f8538146720 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f853000ef90 tx=0x7f853000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f852dffb640 1 -- 172.21.15.138:0/2129959570 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530018020 con 0x7f8538147410 2024-04-25T01:08:08.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f852dffb640 1 -- 172.21.15.138:0/2129959570 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8530014480 con 0x7f8538147410 2024-04-25T01:08:08.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f852dffb640 1 -- 172.21.15.138:0/2129959570 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530014870 con 0x7f8538147410 2024-04-25T01:08:08.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8538145680 con 0x7f8538147410 2024-04-25T01:08:08.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8538145c40 con 0x7f8538147410 2024-04-25T01:08:08.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f852dffb640 1 -- 172.21.15.138:0/2129959570 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f853001b020 con 0x7f8538147410 2024-04-25T01:08:08.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8538069040 con 0x7f8538147410 2024-04-25T01:08:08.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.286+0000 7f852dffb640 1 -- 172.21.15.138:0/2129959570 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f853001dd60 con 0x7f8538147410 2024-04-25T01:08:08.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.290+0000 7f852dffb640 1 -- 172.21.15.138:0/2129959570 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8530022030 con 0x7f8538147410 2024-04-25T01:08:08.286 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:08.294+0000 7fa91274d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:08:08.297 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.302+0000 7f694bd44180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T01:08:08.351 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.358+0000 7f694bd44180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T01:08:08.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.358+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2512be72-afd8-4caf-b076-9c0a07dbac06", "id": 5} v 0) -- 0x7f85380b35e0 con 0x7f8538147410 2024-04-25T01:08:08.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.362+0000 7f852dffb640 1 -- 172.21.15.138:0/2129959570 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2512be72-afd8-4caf-b076-9c0a07dbac06", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f8530010040 con 0x7f8538147410 2024-04-25T01:08:08.358 INFO:teuthology.orchestra.run.smithi138.stdout:5 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8538147410 msgr2=0x7f8538146720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8538147410 0x7f8538146720 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f853000ef90 tx=0x7f853000c490 comp rx=0 tx=0).stop 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 shutdown_connections 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8538147410 0x7f8538146720 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85380b5df0 0x7f853815f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 --2- 172.21.15.138:0/2129959570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85380b4230 0x7f853815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 >> 172.21.15.138:0/2129959570 conn(0x7f85380b2820 msgr2=0x7f85380b1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 shutdown_connections 2024-04-25T01:08:08.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.366+0000 7f853f65e640 1 -- 172.21.15.138:0/2129959570 wait complete. 2024-04-25T01:08:08.374 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 44142ad5-e804-4165-bcb2-c266a808a644 6 2024-04-25T01:08:08.443 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:08.450+0000 7fa91274d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:08:08.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5528554640 1 Processor -- start 2024-04-25T01:08:08.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5528554640 1 -- start start 2024-04-25T01:08:08.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5528554640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55200ce640 0x7f55200d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5528554640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f55200cfb30 0x7f55200cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5528554640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55200cff90 con 0x7f55200d1bc0 2024-04-25T01:08:08.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5528554640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f55200b32b0 con 0x7f55200cfb30 2024-04-25T01:08:08.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5528554640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f55200cef60 con 0x7f55200ce640 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f55262f0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f55200ce640 0x7f55200d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5526af1640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f55200d1bc0 0x7f55200d2030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42138/0 (socket says 172.21.15.138:42138) 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5526af1640 1 -- 172.21.15.138:0/579741418 learned_addr learned my addr 172.21.15.138:0/579741418 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f5525aef640 1 --2- 172.21.15.138:0/579741418 >> v2:172.21.15.33:3300/0 conn(0x7f55200cfb30 0x7f55200cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f55252ee640 1 -- 172.21.15.138:0/579741418 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2248853216 0 0) 0x7f55200cff90 con 0x7f55200d1bc0 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f55252ee640 1 -- 172.21.15.138:0/579741418 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f550c003530 con 0x7f55200d1bc0 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f55262f0640 1 -- 172.21.15.138:0/579741418 >> v2:172.21.15.33:3300/0 conn(0x7f55200cfb30 msgr2=0x7f55200cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f55262f0640 1 --2- 172.21.15.138:0/579741418 >> v2:172.21.15.33:3300/0 conn(0x7f55200cfb30 0x7f55200cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f55262f0640 1 -- 172.21.15.138:0/579741418 >> v1:172.21.15.138:6789/0 conn(0x7f55200d1bc0 legacy=0x7f55200d2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.498+0000 7f55262f0640 1 -- 172.21.15.138:0/579741418 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552015d080 con 0x7f55200ce640 2024-04-25T01:08:08.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f55262f0640 1 --2- 172.21.15.138:0/579741418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200ce640 0x7f55200d1420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f552006eeb0 tx=0x7f551c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19253412ce6721f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f55252ee640 1 -- 172.21.15.138:0/579741418 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c0b1070 con 0x7f55200ce640 2024-04-25T01:08:08.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f55252ee640 1 -- 172.21.15.138:0/579741418 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551c0ac5f0 con 0x7f55200ce640 2024-04-25T01:08:08.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f55252ee640 1 -- 172.21.15.138:0/579741418 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c0ac9c0 con 0x7f55200ce640 2024-04-25T01:08:08.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 -- 172.21.15.138:0/579741418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200ce640 msgr2=0x7f55200d1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 --2- 172.21.15.138:0/579741418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200ce640 0x7f55200d1420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f552006eeb0 tx=0x7f551c01c480 comp rx=0 tx=0).stop 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 -- 172.21.15.138:0/579741418 shutdown_connections 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 --2- 172.21.15.138:0/579741418 >> v2:172.21.15.33:3300/0 conn(0x7f55200cfb30 0x7f55200cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 --2- 172.21.15.138:0/579741418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200ce640 0x7f55200d1420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 -- 172.21.15.138:0/579741418 >> 172.21.15.138:0/579741418 conn(0x7f552009a9d0 msgr2=0x7f552009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 -- 172.21.15.138:0/579741418 shutdown_connections 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 -- 172.21.15.138:0/579741418 wait complete. 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 Processor -- start 2024-04-25T01:08:08.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.502+0000 7f5528554640 1 -- start start 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200ce640 0x7f552009bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200cfb30 0x7f5520150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55200d1bc0 0x7f552006fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5520159ef0 con 0x7f55200ce640 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5520159d70 con 0x7f55200cfb30 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f55200b23e0 con 0x7f55200d1bc0 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5525aef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200cfb30 0x7f5520150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5525aef640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200cfb30 0x7f5520150010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60010/0 (socket says 172.21.15.138:60010) 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5525aef640 1 -- 172.21.15.138:0/1040821364 learned_addr learned my addr 172.21.15.138:0/1040821364 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:08.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5526af1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55200d1bc0 0x7f552006fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f55262f0640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200ce640 0x7f552009bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5526af1640 1 -- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200cfb30 msgr2=0x7f5520150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5526af1640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200cfb30 0x7f5520150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5526af1640 1 -- 172.21.15.138:0/1040821364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200ce640 msgr2=0x7f552009bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5526af1640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200ce640 0x7f552009bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5526af1640 1 -- 172.21.15.138:0/1040821364 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55200b67e0 con 0x7f55200d1bc0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5525aef640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200cfb30 0x7f5520150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5526af1640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55200d1bc0 0x7f552006fae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f551800caa0 tx=0x7f551800cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5516ffd640 1 -- 172.21.15.138:0/1040821364 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518017070 con 0x7f55200d1bc0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5516ffd640 1 -- 172.21.15.138:0/1040821364 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551800ab40 con 0x7f55200d1bc0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5516ffd640 1 -- 172.21.15.138:0/1040821364 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5518012400 con 0x7f55200d1bc0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552009c070 con 0x7f55200d1bc0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f55200b55c0 con 0x7f55200d1bc0 2024-04-25T01:08:08.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5516ffd640 1 -- 172.21.15.138:0/1040821364 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f551801a070 con 0x7f55200d1bc0 2024-04-25T01:08:08.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5516ffd640 1 -- 172.21.15.138:0/1040821364 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f551801ccb0 con 0x7f55200d1bc0 2024-04-25T01:08:08.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.506+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5520069040 con 0x7f55200d1bc0 2024-04-25T01:08:08.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.510+0000 7f5516ffd640 1 -- 172.21.15.138:0/1040821364 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f551801c6c0 con 0x7f55200d1bc0 2024-04-25T01:08:08.507 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:08.514+0000 7fa91274d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:08:08.510 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.518+0000 7f694bd44180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T01:08:08.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.574+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "44142ad5-e804-4165-bcb2-c266a808a644", "id": 6} v 0) -- 0x7f5520152ef0 con 0x7f55200d1bc0 2024-04-25T01:08:08.573 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:08.578+0000 7f694bd44180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T01:08:08.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.582+0000 7f5516ffd640 1 -- 172.21.15.138:0/1040821364 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "44142ad5-e804-4165-bcb2-c266a808a644", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f551801c970 con 0x7f55200d1bc0 2024-04-25T01:08:08.577 INFO:teuthology.orchestra.run.smithi138.stdout:6 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55200d1bc0 msgr2=0x7f552006fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55200d1bc0 0x7f552006fae0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f551800caa0 tx=0x7f551800cea0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 shutdown_connections 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55200d1bc0 0x7f552006fae0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55200cfb30 0x7f5520150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 --2- 172.21.15.138:0/1040821364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55200ce640 0x7f552009bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 >> 172.21.15.138:0/1040821364 conn(0x7f552009a9d0 msgr2=0x7f552009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 shutdown_connections 2024-04-25T01:08:08.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.586+0000 7f5528554640 1 -- 172.21.15.138:0/1040821364 wait complete. 2024-04-25T01:08:08.594 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd new 7cdb7435-2684-4dec-9321-53d1cb275222 7 2024-04-25T01:08:08.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f71cad640 1 Processor -- start 2024-04-25T01:08:08.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f71cad640 1 -- start start 2024-04-25T01:08:08.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f71cad640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8f6c15e5a0 0x7f8f6c0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f71cad640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c0b1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f71cad640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6c06b6f0 con 0x7f8f6c06c950 2024-04-25T01:08:08.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f71cad640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6c06b4a0 con 0x7f8f6c15e5a0 2024-04-25T01:08:08.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f71cad640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c0b54b0 con 0x7f8f6c0b2980 2024-04-25T01:08:08.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c0b1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c0b1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53794/0 (socket says 172.21.15.138:53794) 2024-04-25T01:08:08.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6affd640 1 -- 172.21.15.138:0/202957168 learned_addr learned my addr 172.21.15.138:0/202957168 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:08.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6b7fe640 1 --2- 172.21.15.138:0/202957168 >> v2:172.21.15.33:3300/0 conn(0x7f8f6c15e5a0 0x7f8f6c0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6affd640 1 -- 172.21.15.138:0/202957168 >> v2:172.21.15.33:3300/0 conn(0x7f8f6c15e5a0 msgr2=0x7f8f6c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:08.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6affd640 1 --2- 172.21.15.138:0/202957168 >> v2:172.21.15.33:3300/0 conn(0x7f8f6c15e5a0 0x7f8f6c0b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6affd640 1 -- 172.21.15.138:0/202957168 >> v1:172.21.15.33:6789/0 conn(0x7f8f6c06c950 legacy=0x7f8f6c0ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:08.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.722+0000 7f8f6affd640 1 -- 172.21.15.138:0/202957168 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c06b9d0 con 0x7f8f6c0b2980 2024-04-25T01:08:08.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 --2- 172.21.15.138:0/202957168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c0b1260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009870 tx=0x7f8f5c01c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4346d1d4b001701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6a7fc640 1 -- 172.21.15.138:0/202957168 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0b0070 con 0x7f8f6c0b2980 2024-04-25T01:08:08.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6a7fc640 1 -- 172.21.15.138:0/202957168 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c0ab600 con 0x7f8f6c0b2980 2024-04-25T01:08:08.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6a7fc640 1 -- 172.21.15.138:0/202957168 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0ab9d0 con 0x7f8f6c0b2980 2024-04-25T01:08:08.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- 172.21.15.138:0/202957168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 msgr2=0x7f8f6c0b1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 --2- 172.21.15.138:0/202957168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c0b1260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009870 tx=0x7f8f5c01c520 comp rx=0 tx=0).stop 2024-04-25T01:08:08.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- 172.21.15.138:0/202957168 shutdown_connections 2024-04-25T01:08:08.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 --2- 172.21.15.138:0/202957168 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c0b1260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 --2- 172.21.15.138:0/202957168 >> v2:172.21.15.33:3300/0 conn(0x7f8f6c15e5a0 0x7f8f6c0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- 172.21.15.138:0/202957168 >> 172.21.15.138:0/202957168 conn(0x7f8f6c0b1e60 msgr2=0x7f8f6c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- 172.21.15.138:0/202957168 shutdown_connections 2024-04-25T01:08:08.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- 172.21.15.138:0/202957168 wait complete. 2024-04-25T01:08:08.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 Processor -- start 2024-04-25T01:08:08.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- start start 2024-04-25T01:08:08.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c06c950 0x7f8f6c0994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c15bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c15e5a0 0x7f8f6c15c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c168510 con 0x7f8f6c06c950 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c168390 con 0x7f8f6c15e5a0 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c168690 con 0x7f8f6c0b2980 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c15bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c15bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.138:53802/0 (socket says 172.21.15.138:53802) 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c15e5a0 0x7f8f6c15c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 -- 172.21.15.138:0/1589045514 learned_addr learned my addr 172.21.15.138:0/1589045514 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 -- 172.21.15.138:0/1589045514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c15e5a0 msgr2=0x7f8f6c15c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6b7fe640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c06c950 0x7f8f6c0994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c15e5a0 0x7f8f6c15c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 -- 172.21.15.138:0/1589045514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c06c950 msgr2=0x7f8f6c0994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c06c950 0x7f8f6c0994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 -- 172.21.15.138:0/1589045514 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c099a00 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f6affd640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c15bd70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0ae040 tx=0x7f8f5c01ad60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f63fff640 1 -- 172.21.15.138:0/1589045514 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0b0040 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f63fff640 1 -- 172.21.15.138:0/1589045514 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c0bc070 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c0ad510 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f63fff640 1 -- 172.21.15.138:0/1589045514 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c0b6680 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.726+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c0adad0 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.730+0000 7f8f63fff640 1 -- 172.21.15.138:0/1589045514 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8f5c0b6820 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.730+0000 7f8f63fff640 1 -- 172.21.15.138:0/1589045514 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f8f5c0c03c0 con 0x7f8f6c0b2980 2024-04-25T01:08:08.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.730+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6c069040 con 0x7f8f6c0b2980 2024-04-25T01:08:08.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.730+0000 7f8f63fff640 1 -- 172.21.15.138:0/1589045514 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f8f5c016110 con 0x7f8f6c0b2980 2024-04-25T01:08:08.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.798+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7cdb7435-2684-4dec-9321-53d1cb275222", "id": 7} v 0) -- 0x7f8f6c072130 con 0x7f8f6c0b2980 2024-04-25T01:08:08.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.806+0000 7f8f63fff640 1 -- 172.21.15.138:0/1589045514 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7cdb7435-2684-4dec-9321-53d1cb275222", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8f5c0c1460 con 0x7f8f6c0b2980 2024-04-25T01:08:08.801 INFO:teuthology.orchestra.run.smithi138.stdout:7 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 msgr2=0x7f8f6c15bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c15bd70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0ae040 tx=0x7f8f5c01ad60 comp rx=0 tx=0).stop 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 shutdown_connections 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c15e5a0 0x7f8f6c15c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8f6c0b2980 0x7f8f6c15bd70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 --2- 172.21.15.138:0/1589045514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c06c950 0x7f8f6c0994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 >> 172.21.15.138:0/1589045514 conn(0x7f8f6c0b1e60 msgr2=0x7f8f6c06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 shutdown_connections 2024-04-25T01:08:08.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:08.810+0000 7f8f71cad640 1 -- 172.21.15.138:0/1589045514 wait complete. 2024-04-25T01:08:08.818 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T01:08:08.818 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T01:08:08.821 INFO:tasks.ceph.osd.0:Started 2024-04-25T01:08:08.821 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T01:08:08.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T01:08:08.824 INFO:tasks.ceph.osd.1:Started 2024-04-25T01:08:08.824 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T01:08:08.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T01:08:08.826 INFO:tasks.ceph.osd.2:Started 2024-04-25T01:08:08.826 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T01:08:08.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T01:08:08.829 INFO:tasks.ceph.osd.3:Started 2024-04-25T01:08:08.829 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T01:08:08.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T01:08:08.831 INFO:tasks.ceph.osd.4:Started 2024-04-25T01:08:08.832 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T01:08:08.832 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T01:08:08.834 INFO:tasks.ceph.osd.5:Started 2024-04-25T01:08:08.834 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T01:08:08.834 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T01:08:08.836 INFO:tasks.ceph.osd.6:Started 2024-04-25T01:08:08.836 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T01:08:08.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T01:08:08.840 INFO:tasks.ceph.osd.7:Started 2024-04-25T01:08:08.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 Processor -- start 2024-04-25T01:08:09.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 -- start start 2024-04-25T01:08:09.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f335c072950 0x7f335c067070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f335c0b5e90 0x7f335c072d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f335c0cdb80 con 0x7f335c0b5e90 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f335c06b970 con 0x7f335c072950 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f336301c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f335c073270 con 0x7f335c0aff80 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f335c0b5e90 0x7f335c072d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f335c0b5e90 0x7f335c072d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51576/0 (socket says 172.21.15.33:51576) 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 -- 172.21.15.33:0/3066713021 learned_addr learned my addr 172.21.15.33:0/3066713021 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f3360db8640 1 --2- 172.21.15.33:0/3066713021 >> v2:172.21.15.138:3300/0 conn(0x7f335c072950 0x7f335c067070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f33615b9640 1 --2- 172.21.15.33:0/3066713021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 -- 172.21.15.33:0/3066713021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 msgr2=0x7f335c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 --2- 172.21.15.33:0/3066713021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 -- 172.21.15.33:0/3066713021 >> v2:172.21.15.138:3300/0 conn(0x7f335c072950 msgr2=0x7f335c067070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:09.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 --2- 172.21.15.33:0/3066713021 >> v2:172.21.15.138:3300/0 conn(0x7f335c072950 0x7f335c067070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.030+0000 7f335bfff640 1 -- 172.21.15.33:0/3066713021 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c06b3b0 con 0x7f335c0b5e90 2024-04-25T01:08:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f335bfff640 1 --2- 172.21.15.33:0/3066713021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c0b5e90 0x7f335c072d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f335c06cd20 tx=0x7f334801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47109cfa8b22df67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:09.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f335b7fe640 1 -- 172.21.15.33:0/3066713021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480aa610 con 0x7f335c0b5e90 2024-04-25T01:08:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f335b7fe640 1 -- 172.21.15.33:0/3066713021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33480af070 con 0x7f335c0b5e90 2024-04-25T01:08:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f335b7fe640 1 -- 172.21.15.33:0/3066713021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480aa9d0 con 0x7f335c0b5e90 2024-04-25T01:08:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- 172.21.15.33:0/3066713021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c0b5e90 msgr2=0x7f335c072d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:09.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 --2- 172.21.15.33:0/3066713021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c0b5e90 0x7f335c072d30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f335c06cd20 tx=0x7f334801c4f0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- 172.21.15.33:0/3066713021 shutdown_connections 2024-04-25T01:08:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 --2- 172.21.15.33:0/3066713021 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 --2- 172.21.15.33:0/3066713021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c0b5e90 0x7f335c072d30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 --2- 172.21.15.33:0/3066713021 >> v2:172.21.15.138:3300/0 conn(0x7f335c072950 0x7f335c067070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- 172.21.15.33:0/3066713021 >> 172.21.15.33:0/3066713021 conn(0x7f335c0b50c0 msgr2=0x7f335c0b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- 172.21.15.33:0/3066713021 shutdown_connections 2024-04-25T01:08:09.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- 172.21.15.33:0/3066713021 wait complete. 2024-04-25T01:08:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 Processor -- start 2024-04-25T01:08:09.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- start start 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c072950 0x7f335c14c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c0b5e90 0x7f335c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f335c0b0d60 con 0x7f335c072950 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f335c15e4e0 con 0x7f335c0b5e90 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f336301c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f335c154770 con 0x7f335c0aff80 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c072950 0x7f335c14c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c072950 0x7f335c14c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51592/0 (socket says 172.21.15.33:51592) 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 -- 172.21.15.33:0/49760302 learned_addr learned my addr 172.21.15.33:0/49760302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f335bfff640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f33615b9640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c0b5e90 0x7f335c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 -- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 msgr2=0x7f335c152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:09.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 -- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c0b5e90 msgr2=0x7f335c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c0b5e90 0x7f335c15c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 -- 172.21.15.33:0/49760302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335c14c800 con 0x7f335c072950 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f3360db8640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c072950 0x7f335c14c2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f335000e8b0 tx=0x7f335000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.034+0000 7f33597fa640 1 -- 172.21.15.33:0/49760302 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3350017070 con 0x7f335c072950 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f33597fa640 1 -- 172.21.15.33:0/49760302 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f335000c8e0 con 0x7f335c072950 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335c1536d0 con 0x7f335c072950 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f33597fa640 1 -- 172.21.15.33:0/49760302 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335000ccb0 con 0x7f335c072950 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f335c153f70 con 0x7f335c072950 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f33597fa640 1 -- 172.21.15.33:0/49760302 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f335001a070 con 0x7f335c072950 2024-04-25T01:08:09.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f33597fa640 1 -- 172.21.15.33:0/49760302 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f3350013b40 con 0x7f335c072950 2024-04-25T01:08:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335c153ac0 con 0x7f335c072950 2024-04-25T01:08:09.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.038+0000 7f33597fa640 1 -- 172.21.15.33:0/49760302 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f335c14c800 con 0x7f335c072950 2024-04-25T01:08:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.094+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f335c069040 con 0x7f335c072950 2024-04-25T01:08:09.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.094+0000 7f33597fa640 1 -- 172.21.15.33:0/49760302 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f3350013e20 con 0x7f335c072950 2024-04-25T01:08:09.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:09.088 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":10,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:08.803308+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c072950 msgr2=0x7f335c14c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c072950 0x7f335c14c2c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f335000e8b0 tx=0x7f335000ed70 comp rx=0 tx=0).stop 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 shutdown_connections 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f335c0b5e90 0x7f335c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f335c0aff80 0x7f335c152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 --2- 172.21.15.33:0/49760302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f335c072950 0x7f335c14c2c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 >> 172.21.15.33:0/49760302 conn(0x7f335c0b50c0 msgr2=0x7f335c14e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 shutdown_connections 2024-04-25T01:08:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:09.098+0000 7f336301c640 1 -- 172.21.15.33:0/49760302 wait complete. 2024-04-25T01:08:09.103 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T01:08:09.103 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T01:08:09.199 INFO:tasks.ceph.mgr.x.smithi138.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T01:08:09.199 INFO:tasks.ceph.mgr.x.smithi138.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T01:08:09.199 INFO:tasks.ceph.mgr.x.smithi138.stderr: from numpy import show_config as show_numpy_config 2024-04-25T01:08:09.201 INFO:tasks.ceph.mgr.y.smithi033.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T01:08:09.201 INFO:tasks.ceph.mgr.y.smithi033.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T01:08:09.201 INFO:tasks.ceph.mgr.y.smithi033.stderr: from numpy import show_config as show_numpy_config 2024-04-25T01:08:09.207 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T01:08:09.214+0000 7ff6eb5137c0 -1 Falling back to public interface 2024-04-25T01:08:09.207 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T01:08:09.214+0000 7fb041d257c0 -1 Falling back to public interface 2024-04-25T01:08:09.211 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T01:08:09.218+0000 7f2a078e17c0 -1 Falling back to public interface 2024-04-25T01:08:09.223 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T01:08:09.230+0000 7f502df4b7c0 -1 Falling back to public interface 2024-04-25T01:08:09.240 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:09.246+0000 7f694bd44180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:08:09.273 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:09.282+0000 7fa91274d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T01:08:09.372 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:09.378+0000 7f694bd44180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:08:09.440 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:09.446+0000 7fa91274d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:08:09.478 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-04-25T01:08:09.486+0000 7f10080e07c0 -1 Falling back to public interface 2024-04-25T01:08:09.493 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-04-25T01:08:09.502+0000 7f1f34c017c0 -1 Falling back to public interface 2024-04-25T01:08:09.494 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-04-25T01:08:09.502+0000 7fad005887c0 -1 Falling back to public interface 2024-04-25T01:08:09.505 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-04-25T01:08:09.514+0000 7f3814dff7c0 -1 Falling back to public interface 2024-04-25T01:08:09.544 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:09.550+0000 7f694bd44180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:08:09.607 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:09.614+0000 7f694bd44180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:08:09.625 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:09.634+0000 7fa91274d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T01:08:09.697 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:09.706+0000 7fa91274d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T01:08:09.741 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:09.746+0000 7f694bd44180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:08:09.847 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:09.854+0000 7fa91274d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T01:08:09.909 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:10.016 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf12e8f640 1 Processor -- start 2024-04-25T01:08:10.018 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf12e8f640 1 -- start start 2024-04-25T01:08:10.018 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf12e8f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf0c06c290 0x7fcf0c0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:10.018 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf12e8f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c0af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:10.018 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf12e8f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0c06b510 con 0x7fcf0c15a570 2024-04-25T01:08:10.018 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf12e8f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fcf0c0c7010 con 0x7fcf0c06c290 2024-04-25T01:08:10.018 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf12e8f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c0c6490 con 0x7fcf0c06d410 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf0c06c290 0x7fcf0c0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcf0c06c290 0x7fcf0c0c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51608/0 (socket says 172.21.15.33:51608) 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf10c2b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcf0c15a570 0x7fcf0c06bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45950/0 (socket says 172.21.15.33:45950) 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 -- 172.21.15.33:0/2901151049 learned_addr learned my addr 172.21.15.33:0/2901151049 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf1142c640 1 --2- 172.21.15.33:0/2901151049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c0af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 -- 172.21.15.33:0/2901151049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 msgr2=0x7fcf0c0af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 --2- 172.21.15.33:0/2901151049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c0af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 -- 172.21.15.33:0/2901151049 >> v1:172.21.15.33:6789/0 conn(0x7fcf0c15a570 legacy=0x7fcf0c06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 -- 172.21.15.33:0/2901151049 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c0c6770 con 0x7fcf0c06c290 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2901151049 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4113822160 0 0) 0x7fcf0c06b510 con 0x7fcf0c15a570 2024-04-25T01:08:10.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf1142c640 1 --2- 172.21.15.33:0/2901151049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c0af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:10.020 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0bfff640 1 --2- 172.21.15.33:0/2901151049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c06c290 0x7fcf0c0c5e90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009870 tx=0x7fcef801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9c09555be4d8a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:10.020 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2901151049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80ab650 con 0x7fcf0c06c290 2024-04-25T01:08:10.020 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2901151049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef80b0070 con 0x7fcf0c06c290 2024-04-25T01:08:10.020 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.026+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2901151049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef80aba10 con 0x7fcf0c06c290 2024-04-25T01:08:10.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- 172.21.15.33:0/2901151049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c06c290 msgr2=0x7fcf0c0c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:10.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 --2- 172.21.15.33:0/2901151049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c06c290 0x7fcf0c0c5e90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009870 tx=0x7fcef801c560 comp rx=0 tx=0).stop 2024-04-25T01:08:10.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- 172.21.15.33:0/2901151049 shutdown_connections 2024-04-25T01:08:10.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 --2- 172.21.15.33:0/2901151049 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c0af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 --2- 172.21.15.33:0/2901151049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c06c290 0x7fcf0c0c5e90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- 172.21.15.33:0/2901151049 >> 172.21.15.33:0/2901151049 conn(0x7fcf0c0b1dc0 msgr2=0x7fcf0c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:10.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- 172.21.15.33:0/2901151049 shutdown_connections 2024-04-25T01:08:10.022 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- 172.21.15.33:0/2901151049 wait complete. 2024-04-25T01:08:10.022 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 Processor -- start 2024-04-25T01:08:10.022 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- start start 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c06c290 0x7fcf0c0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c15a570 0x7fcf0c0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c0ccff0 con 0x7fcf0c15a570 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0c15ab60 con 0x7fcf0c06c290 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf12e8f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0c0cd170 con 0x7fcf0c06d410 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c15a570 0x7fcf0c0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c15a570 0x7fcf0c0c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51618/0 (socket says 172.21.15.33:51618) 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 -- 172.21.15.33:0/3931413717 learned_addr learned my addr 172.21.15.33:0/3931413717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf0bfff640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf10c2b640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c06c290 0x7fcf0c0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:10.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 -- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 msgr2=0x7fcf0c153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 -- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c06c290 msgr2=0x7fcf0c0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c06c290 0x7fcf0c0ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 -- 172.21.15.33:0/3931413717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf0c1f48a0 con 0x7fcf0c15a570 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.030+0000 7fcf1142c640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c15a570 0x7fcf0c0c7560 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400cef0 tx=0x7fcf0400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf097fa640 1 -- 172.21.15.33:0/3931413717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0401a070 con 0x7fcf0c15a570 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf097fa640 1 -- 172.21.15.33:0/3931413717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0400d860 con 0x7fcf0c15a570 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0c0cabc0 con 0x7fcf0c15a570 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf097fa640 1 -- 172.21.15.33:0/3931413717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400dc50 con 0x7fcf0c15a570 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0c0cb180 con 0x7fcf0c15a570 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf097fa640 1 -- 172.21.15.33:0/3931413717 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf0401d070 con 0x7fcf0c15a570 2024-04-25T01:08:10.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf097fa640 1 -- 172.21.15.33:0/3931413717 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcf0400ddf0 con 0x7fcf0c15a570 2024-04-25T01:08:10.027 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0c1f4a70 con 0x7fcf0c15a570 2024-04-25T01:08:10.027 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.034+0000 7fcf097fa640 1 -- 172.21.15.33:0/3931413717 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcf04024020 con 0x7fcf0c15a570 2024-04-25T01:08:10.085 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf0c0b00b0 con 0x7fcf0c15a570 2024-04-25T01:08:10.085 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf097fa640 1 -- 172.21.15.33:0/3931413717 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcf0401f550 con 0x7fcf0c15a570 2024-04-25T01:08:10.085 INFO:teuthology.misc.health.smithi033.stdout: 2024-04-25T01:08:10.085 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:08.803308+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c15a570 msgr2=0x7fcf0c0c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c15a570 0x7fcf0c0c7560 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400cef0 tx=0x7fcf0400a710 comp rx=0 tx=0).stop 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 shutdown_connections 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf0c15a570 0x7fcf0c0c7560 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcf0c06d410 0x7fcf0c153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 --2- 172.21.15.33:0/3931413717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf0c06c290 0x7fcf0c0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 >> 172.21.15.33:0/3931413717 conn(0x7fcf0c0b1dc0 msgr2=0x7fcf0c074680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 shutdown_connections 2024-04-25T01:08:10.087 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:10.094+0000 7fcf12e8f640 1 -- 172.21.15.33:0/3931413717 wait complete. 2024-04-25T01:08:10.101 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T01:08:10.361 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:10.366+0000 7f694bd44180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:08:10.436 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:10.442+0000 7f694bd44180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:08:10.478 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:10.486+0000 7fa91274d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T01:08:10.557 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:10.566+0000 7fa91274d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T01:08:10.618 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:10.622+0000 7f694bd44180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:08:10.692 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:10.698+0000 7f694bd44180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:08:10.746 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:10.754+0000 7fa91274d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T01:08:10.823 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:10.830+0000 7fa91274d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T01:08:10.981 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:10.986+0000 7f694bd44180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:08:11.043 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:11.050+0000 7f694bd44180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:08:11.131 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:11.138+0000 7fa91274d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T01:08:11.196 INFO:tasks.ceph.mgr.y.smithi033.stderr:2024-04-25T01:08:11.202+0000 7fa91274d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T01:08:12.090 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:12.098+0000 7f68f3191640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T01:08:12.090 INFO:tasks.ceph.mgr.x.smithi138.stderr:2024-04-25T01:08:12.098+0000 7f68f3191640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T01:08:12.868 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-04-25T01:08:12.878+0000 7f3814dff7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T01:08:12.984 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T01:08:12.990+0000 7fb041d257c0 -1 osd.5 0 log_to_monitors true 2024-04-25T01:08:13.119 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T01:08:13.126+0000 7f502df4b7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T01:08:13.129 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T01:08:13.134+0000 7ff6eb5137c0 -1 osd.7 0 log_to_monitors true 2024-04-25T01:08:13.136 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T01:08:13.142+0000 7f2a078e17c0 -1 osd.4 0 log_to_monitors true 2024-04-25T01:08:13.298 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-04-25T01:08:13.306+0000 7f10080e07c0 -1 osd.2 0 log_to_monitors true 2024-04-25T01:08:13.405 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-04-25T01:08:13.414+0000 7fad005887c0 -1 osd.3 0 log_to_monitors true 2024-04-25T01:08:13.418 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-04-25T01:08:13.426+0000 7f1f34c017c0 -1 osd.1 0 log_to_monitors true 2024-04-25T01:08:14.935 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T01:08:14.942+0000 7fb03dcce640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T01:08:14.959 INFO:tasks.ceph.osd.5.smithi138.stderr:2024-04-25T01:08:14.966+0000 7fb035b03640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:14.966 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-04-25T01:08:14.974+0000 7f3810da8640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T01:08:14.990 INFO:tasks.ceph.osd.0.smithi033.stderr:2024-04-25T01:08:14.998+0000 7f3808bdd640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:16.836 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T01:08:16.842+0000 7f2a0409c640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T01:08:16.854 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-04-25T01:08:16.862+0000 7f1f30baa640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T01:08:16.862 INFO:tasks.ceph.osd.4.smithi138.stderr:2024-04-25T01:08:16.870+0000 7f29fb6bf640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:16.870 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T01:08:16.878+0000 7ff6e7cce640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T01:08:16.876 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-04-25T01:08:16.886+0000 7f1004089640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T01:08:16.877 INFO:tasks.ceph.osd.1.smithi033.stderr:2024-04-25T01:08:16.886+0000 7f1f289df640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:16.887 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T01:08:16.894+0000 7f5029ef4640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T01:08:16.894 INFO:tasks.ceph.osd.7.smithi138.stderr:2024-04-25T01:08:16.902+0000 7ff6df2f1640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:16.905 INFO:tasks.ceph.osd.2.smithi033.stderr:2024-04-25T01:08:16.914+0000 7f0ffbebe640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:16.908 DEBUG:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:16.928 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-04-25T01:08:16.934+0000 7facfcd43640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T01:08:16.931 INFO:tasks.ceph.osd.6.smithi138.stderr:2024-04-25T01:08:16.938+0000 7f5021d29640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:16.949 INFO:tasks.ceph.osd.3.smithi033.stderr:2024-04-25T01:08:16.958+0000 7facf4366640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:08:17.019 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.026+0000 7ff73505a640 1 Processor -- start 2024-04-25T01:08:17.020 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.026+0000 7ff73505a640 1 -- start start 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff73505a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7300cb950 0x7ff73015c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff73505a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7300cc960 con 0x7ff730071760 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff73505a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7300cbde0 con 0x7ff730070b50 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff73505a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff7300cc0c0 con 0x7ff7300cb950 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72e59c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff7300cb950 0x7ff73015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72f59e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff730071760 0x7ff730072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45962/0 (socket says 172.21.15.33:45962) 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72f59e640 1 -- 172.21.15.33:0/138445153 learned_addr learned my addr 172.21.15.33:0/138445153 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:17.021 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72e59c640 1 --2- 172.21.15.33:0/138445153 >> v2:172.21.15.33:3300/0 conn(0x7ff7300cb950 0x7ff73015c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51670/0 (socket says 172.21.15.33:51670) 2024-04-25T01:08:17.022 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72e59c640 1 -- 172.21.15.33:0/138445153 >> v1:172.21.15.138:6789/0 conn(0x7ff730070b50 legacy=0x7ff730070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.022 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72e59c640 1 -- 172.21.15.33:0/138445153 >> v1:172.21.15.33:6789/0 conn(0x7ff730071760 legacy=0x7ff730072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.022 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72e59c640 1 -- 172.21.15.33:0/138445153 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7300cc3a0 con 0x7ff7300cb950 2024-04-25T01:08:17.023 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72e59c640 1 --2- 172.21.15.33:0/138445153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300cb950 0x7ff73015c190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff7240097c0 tx=0x7ff72401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b58b711ad66aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.024 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72dd9b640 1 -- 172.21.15.33:0/138445153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240b0070 con 0x7ff7300cb950 2024-04-25T01:08:17.024 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72dd9b640 1 -- 172.21.15.33:0/138445153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7240ab600 con 0x7ff7300cb950 2024-04-25T01:08:17.024 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.030+0000 7ff72dd9b640 1 -- 172.21.15.33:0/138445153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240ab9d0 con 0x7ff7300cb950 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- 172.21.15.33:0/138445153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300cb950 msgr2=0x7ff73015c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 --2- 172.21.15.33:0/138445153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300cb950 0x7ff73015c190 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff7240097c0 tx=0x7ff72401c330 comp rx=0 tx=0).stop 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- 172.21.15.33:0/138445153 shutdown_connections 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 --2- 172.21.15.33:0/138445153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7300cb950 0x7ff73015c190 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- 172.21.15.33:0/138445153 >> 172.21.15.33:0/138445153 conn(0x7ff73006b210 msgr2=0x7ff73006b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- 172.21.15.33:0/138445153 shutdown_connections 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- 172.21.15.33:0/138445153 wait complete. 2024-04-25T01:08:17.025 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 Processor -- start 2024-04-25T01:08:17.026 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- start start 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730070b50 0x7ff7301f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730071760 0x7ff7301f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7301e4490 0x7ff7300cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7300cdde0 con 0x7ff730071760 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7300ce2a0 con 0x7ff730070b50 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff7300cdf60 con 0x7ff7301e4490 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730071760 0x7ff7301f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730071760 0x7ff7301f8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51680/0 (socket says 172.21.15.33:51680) 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 -- 172.21.15.33:0/925600211 learned_addr learned my addr 172.21.15.33:0/925600211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:17.028 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 -- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7301e4490 msgr2=0x7ff7300cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72f59e640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7301e4490 0x7ff7300cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72ed9d640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730070b50 0x7ff7301f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7301e4490 0x7ff7300cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 -- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730070b50 msgr2=0x7ff7301f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730070b50 0x7ff7301f4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 -- 172.21.15.33:0/925600211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7301e4910 con 0x7ff730071760 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff72e59c640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730071760 0x7ff7301f8470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff724018f40 tx=0x7ff7240190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff7177fe640 1 -- 172.21.15.33:0/925600211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240b0040 con 0x7ff730071760 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff7177fe640 1 -- 172.21.15.33:0/925600211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7240ba070 con 0x7ff730071760 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7301e4bc0 con 0x7ff730071760 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff7177fe640 1 -- 172.21.15.33:0/925600211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7240aac30 con 0x7ff730071760 2024-04-25T01:08:17.029 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.034+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7301f47b0 con 0x7ff730071760 2024-04-25T01:08:17.030 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.038+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7300686e0 con 0x7ff730071760 2024-04-25T01:08:17.030 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.038+0000 7ff7177fe640 1 -- 172.21.15.33:0/925600211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7ff7240ab420 con 0x7ff730071760 2024-04-25T01:08:17.030 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.038+0000 7ff7177fe640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff71c067830 0x7ff71c069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.031 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.038+0000 7ff7177fe640 1 -- 172.21.15.33:0/925600211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3166+0+0 (secure 0 0 0) 0x7ff72411f440 con 0x7ff730071760 2024-04-25T01:08:17.031 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.038+0000 7ff72ed9d640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff71c067830 0x7ff71c069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.031 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.038+0000 7ff72ed9d640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff71c067830 0x7ff71c069cf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff73015a510 tx=0x7ff720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.034 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.042+0000 7ff7177fe640 1 -- 172.21.15.33:0/925600211 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff724015fb0 con 0x7ff730071760 2024-04-25T01:08:17.211 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.218+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff73004f580 con 0x7ff730071760 2024-04-25T01:08:17.212 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.218+0000 7ff7177fe640 1 -- 172.21.15.33:0/925600211 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+12747 (secure 0 0 0) 0x7ff7240f0a50 con 0x7ff730071760 2024-04-25T01:08:17.212 INFO:teuthology.misc.health.smithi033.stdout: 2024-04-25T01:08:17.212 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":15,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:17.160602+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff71c067830 msgr2=0x7ff71c069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff71c067830 0x7ff71c069cf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff73015a510 tx=0x7ff720009040 comp rx=0 tx=0).stop 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730071760 msgr2=0x7ff7301f8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730071760 0x7ff7301f8470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff724018f40 tx=0x7ff7240190d0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 shutdown_connections 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff71c067830 0x7ff71c069cf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff7301e4490 0x7ff7300cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff730071760 0x7ff7301f8470 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 --2- 172.21.15.33:0/925600211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff730070b50 0x7ff7301f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 >> 172.21.15.33:0/925600211 conn(0x7ff73006b210 msgr2=0x7ff73015a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:17.215 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 shutdown_connections 2024-04-25T01:08:17.216 INFO:teuthology.misc.health.smithi033.stderr:2024-04-25T01:08:17.222+0000 7ff73505a640 1 -- 172.21.15.33:0/925600211 wait complete. 2024-04-25T01:08:17.230 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T01:08:17.230 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T01:08:17.230 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T01:08:17.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T01:08:17.273 INFO:tasks.ceph.mds.a:Started 2024-04-25T01:08:17.273 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T01:08:17.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T01:08:17.276 INFO:tasks.ceph.mds.c:Started 2024-04-25T01:08:17.276 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T01:08:17.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T01:08:17.279 INFO:tasks.ceph.mds.b:Started 2024-04-25T01:08:17.279 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T01:08:17.279 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T01:08:17.282 INFO:tasks.ceph.mds.d:Started 2024-04-25T01:08:17.282 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T01:08:17.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:17.350 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2024-04-25T01:08:17.350 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2024-04-25T01:08:17.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0ee22640 1 Processor -- start 2024-04-25T01:08:17.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0ee22640 1 -- start start 2024-04-25T01:08:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0ee22640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6e080b0420 0x7f6e080b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0ee22640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e080aed70 0x7f6e080af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0ee22640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e080ae2d0 con 0x7f6e0815e010 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0ee22640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6e080afd40 con 0x7f6e080aed70 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0ee22640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6e080b0020 con 0x7f6e080b0420 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0d3bf640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e080aed70 0x7f6e080af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0cbbe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6e0815e010 0x7f6e0806dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:45994/0 (socket says 172.21.15.33:45994) 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0cbbe640 1 -- 172.21.15.33:0/618588010 learned_addr learned my addr 172.21.15.33:0/618588010 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6dfffff640 1 --2- 172.21.15.33:0/618588010 >> v2:172.21.15.138:3300/0 conn(0x7f6e080b0420 0x7f6e080b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0d3bf640 1 -- 172.21.15.33:0/618588010 >> v2:172.21.15.138:3300/0 conn(0x7f6e080b0420 msgr2=0x7f6e080b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0d3bf640 1 --2- 172.21.15.33:0/618588010 >> v2:172.21.15.138:3300/0 conn(0x7f6e080b0420 0x7f6e080b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0d3bf640 1 -- 172.21.15.33:0/618588010 >> v1:172.21.15.33:6789/0 conn(0x7f6e0815e010 legacy=0x7f6e0806dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6dff7fe640 1 -- 172.21.15.33:0/618588010 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4000016729 0 0) 0x7f6e080ae2d0 con 0x7f6e0815e010 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6e0d3bf640 1 -- 172.21.15.33:0/618588010 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e080ae570 con 0x7f6e080aed70 2024-04-25T01:08:17.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.406+0000 7f6dfffff640 1 --2- 172.21.15.33:0/618588010 >> v2:172.21.15.138:3300/0 conn(0x7f6e080b0420 0x7f6e080b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0d3bf640 1 --2- 172.21.15.33:0/618588010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e080aed70 0x7f6e080af1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e080ae4a0 tx=0x7f6e0001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa13d7615128672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6dff7fe640 1 -- 172.21.15.33:0/618588010 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000b2070 con 0x7f6e080aed70 2024-04-25T01:08:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6dff7fe640 1 -- 172.21.15.33:0/618588010 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e000a58e0 con 0x7f6e080aed70 2024-04-25T01:08:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6dff7fe640 1 -- 172.21.15.33:0/618588010 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000a5cb0 con 0x7f6e080aed70 2024-04-25T01:08:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 -- 172.21.15.33:0/618588010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e080aed70 msgr2=0x7f6e080af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/618588010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e080aed70 0x7f6e080af1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6e080ae4a0 tx=0x7f6e0001c930 comp rx=0 tx=0).stop 2024-04-25T01:08:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 -- 172.21.15.33:0/618588010 shutdown_connections 2024-04-25T01:08:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/618588010 >> v2:172.21.15.138:3300/0 conn(0x7f6e080b0420 0x7f6e080b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/618588010 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e080aed70 0x7f6e080af1a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 -- 172.21.15.33:0/618588010 >> 172.21.15.33:0/618588010 conn(0x7f6e080b29c0 msgr2=0x7f6e080b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 -- 172.21.15.33:0/618588010 shutdown_connections 2024-04-25T01:08:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 -- 172.21.15.33:0/618588010 wait complete. 2024-04-25T01:08:17.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 Processor -- start 2024-04-25T01:08:17.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.410+0000 7f6e0ee22640 1 -- start start 2024-04-25T01:08:17.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e080aed70 0x7f6e081f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e080b0420 0x7f6e080c6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e0815e010 0x7f6e0806e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0816ae80 con 0x7f6e0815e010 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08072fe0 con 0x7f6e080b0420 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0816b000 con 0x7f6e080aed70 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e0815e010 0x7f6e0806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e0815e010 0x7f6e0806e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51728/0 (socket says 172.21.15.33:51728) 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 -- 172.21.15.33:0/195042613 learned_addr learned my addr 172.21.15.33:0/195042613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0cbbe640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e080aed70 0x7f6e081f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6dfffff640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e080b0420 0x7f6e080c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 -- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e080aed70 msgr2=0x7f6e081f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e080aed70 0x7f6e081f95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 -- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e080b0420 msgr2=0x7f6e080c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e080b0420 0x7f6e080c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 -- 172.21.15.33:0/195042613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e080c7d70 con 0x7f6e0815e010 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0cbbe640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e080aed70 0x7f6e081f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0d3bf640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e0815e010 0x7f6e0806e600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e0001ce10 tx=0x7f6e000196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6dfd7fa640 1 -- 172.21.15.33:0/195042613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e000b2040 con 0x7f6e0815e010 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6dfd7fa640 1 -- 172.21.15.33:0/195042613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e000039a0 con 0x7f6e0815e010 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e080c71f0 con 0x7f6e0815e010 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6dfd7fa640 1 -- 172.21.15.33:0/195042613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00003d90 con 0x7f6e0815e010 2024-04-25T01:08:17.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e080c77b0 con 0x7f6e0815e010 2024-04-25T01:08:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e08069040 con 0x7f6e0815e010 2024-04-25T01:08:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6dfd7fa640 1 -- 172.21.15.33:0/195042613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f6e000c1030 con 0x7f6e0815e010 2024-04-25T01:08:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6dfd7fa640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6df4067830 0x7f6df4069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6dfd7fa640 1 -- 172.21.15.33:0/195042613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f6e001201c0 con 0x7f6e0815e010 2024-04-25T01:08:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.414+0000 7f6e0cbbe640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6df4067830 0x7f6df4069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.418+0000 7f6e0cbbe640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6df4067830 0x7f6df4069cf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e04000f70 tx=0x7f6e04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.418+0000 7f6dfd7fa640 1 -- 172.21.15.33:0/195042613 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6e000c8020 con 0x7f6e0815e010 2024-04-25T01:08:17.503 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-04-25T01:08:17.510 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-04-25T01:08:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.582+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e0806bd40 con 0x7f6e0815e010 2024-04-25T01:08:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6dfd7fa640 1 -- 172.21.15.33:0/195042613 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+12747 (secure 0 0 0) 0x7f6e00016520 con 0x7f6e0815e010 2024-04-25T01:08:17.577 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:17.577 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:17.160602+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6df4067830 msgr2=0x7f6df4069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6df4067830 0x7f6df4069cf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e04000f70 tx=0x7f6e04009040 comp rx=0 tx=0).stop 2024-04-25T01:08:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e0815e010 msgr2=0x7f6e0806e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e0815e010 0x7f6e0806e600 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e0001ce10 tx=0x7f6e000196d0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 shutdown_connections 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6df4067830 0x7f6df4069cf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e0815e010 0x7f6e0806e600 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e080b0420 0x7f6e080c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 --2- 172.21.15.33:0/195042613 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e080aed70 0x7f6e081f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 >> 172.21.15.33:0/195042613 conn(0x7f6e080b29c0 msgr2=0x7f6e0815d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 shutdown_connections 2024-04-25T01:08:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.586+0000 7f6e0ee22640 1 -- 172.21.15.33:0/195042613 wait complete. 2024-04-25T01:08:17.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:08:17.597 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:08:17.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35e1470640 1 Processor -- start 2024-04-25T01:08:17.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35e1470640 1 -- start start 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35e1470640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f35dc15e0d0 0x7f35dc0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35e1470640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f35dc0d1f60 0x7f35dc0d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35e1470640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35dc0b6cc0 con 0x7f35dc0b9a20 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35e1470640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f35dc0b7580 con 0x7f35dc15e0d0 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35e1470640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f35dc0b6f30 con 0x7f35dc0d1f60 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35daffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f35dc15e0d0 0x7f35dc0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35daffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f35dc15e0d0 0x7f35dc0cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51736/0 (socket says 172.21.15.33:51736) 2024-04-25T01:08:17.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35daffd640 1 -- 172.21.15.33:0/1318500555 learned_addr learned my addr 172.21.15.33:0/1318500555 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.734+0000 7f35da7fc640 1 --2- 172.21.15.33:0/1318500555 >> v2:172.21.15.138:3300/0 conn(0x7f35dc0d1f60 0x7f35dc0d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35daffd640 1 -- 172.21.15.33:0/1318500555 >> v2:172.21.15.138:3300/0 conn(0x7f35dc0d1f60 msgr2=0x7f35dc0d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35daffd640 1 --2- 172.21.15.33:0/1318500555 >> v2:172.21.15.138:3300/0 conn(0x7f35dc0d1f60 0x7f35dc0d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35daffd640 1 -- 172.21.15.33:0/1318500555 >> v1:172.21.15.138:6789/0 conn(0x7f35dc0b9a20 legacy=0x7f35dc0d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35daffd640 1 -- 172.21.15.33:0/1318500555 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc0b7210 con 0x7f35dc15e0d0 2024-04-25T01:08:17.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35daffd640 1 --2- 172.21.15.33:0/1318500555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc15e0d0 0x7f35dc0cf3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35dc08e300 tx=0x7f35cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab50f4b1f2bd470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35d9ffb640 1 -- 172.21.15.33:0/1318500555 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0ab430 con 0x7f35dc15e0d0 2024-04-25T01:08:17.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35d9ffb640 1 -- 172.21.15.33:0/1318500555 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35cc0b0070 con 0x7f35dc15e0d0 2024-04-25T01:08:17.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35d9ffb640 1 -- 172.21.15.33:0/1318500555 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0ab7f0 con 0x7f35dc15e0d0 2024-04-25T01:08:17.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35e1470640 1 -- 172.21.15.33:0/1318500555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc15e0d0 msgr2=0x7f35dc0cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.738+0000 7f35e1470640 1 --2- 172.21.15.33:0/1318500555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc15e0d0 0x7f35dc0cf3f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f35dc08e300 tx=0x7f35cc01c520 comp rx=0 tx=0).stop 2024-04-25T01:08:17.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- 172.21.15.33:0/1318500555 shutdown_connections 2024-04-25T01:08:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 --2- 172.21.15.33:0/1318500555 >> v2:172.21.15.138:3300/0 conn(0x7f35dc0d1f60 0x7f35dc0d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 --2- 172.21.15.33:0/1318500555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc15e0d0 0x7f35dc0cf3f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- 172.21.15.33:0/1318500555 >> 172.21.15.33:0/1318500555 conn(0x7f35dc0d3dd0 msgr2=0x7f35dc0d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- 172.21.15.33:0/1318500555 shutdown_connections 2024-04-25T01:08:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- 172.21.15.33:0/1318500555 wait complete. 2024-04-25T01:08:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 Processor -- start 2024-04-25T01:08:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- start start 2024-04-25T01:08:17.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35dc0b9a20 0x7f35dc15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc0d1f60 0x7f35dc09efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35dc15e0d0 0x7f35dc15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc16ae90 con 0x7f35dc0d1f60 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc16ad10 con 0x7f35dc0b9a20 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f35dc16b010 con 0x7f35dc15e0d0 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc0d1f60 0x7f35dc09efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc0d1f60 0x7f35dc09efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51752/0 (socket says 172.21.15.33:51752) 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 -- 172.21.15.33:0/828644960 learned_addr learned my addr 172.21.15.33:0/828644960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35daffd640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35dc0b9a20 0x7f35dc15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35db7fe640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35dc15e0d0 0x7f35dc15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 -- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35dc15e0d0 msgr2=0x7f35dc15fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35dc15e0d0 0x7f35dc15fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 -- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35dc0b9a20 msgr2=0x7f35dc15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35dc0b9a20 0x7f35dc15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 -- 172.21.15.33:0/828644960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc0a07e0 con 0x7f35dc0d1f60 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35da7fc640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc0d1f60 0x7f35dc09efe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e930 tx=0x7f35c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35d37fe640 1 -- 172.21.15.33:0/828644960 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4017070 con 0x7f35dc0d1f60 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35d37fe640 1 -- 172.21.15.33:0/828644960 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c400ca00 con 0x7f35dc0d1f60 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc0a0ac0 con 0x7f35dc0d1f60 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35d37fe640 1 -- 172.21.15.33:0/828644960 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c400cdd0 con 0x7f35dc0d1f60 2024-04-25T01:08:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.742+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc1f7450 con 0x7f35dc0d1f60 2024-04-25T01:08:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.746+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35dc069040 con 0x7f35dc0d1f60 2024-04-25T01:08:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.746+0000 7f35d37fe640 1 -- 172.21.15.33:0/828644960 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f35c401a070 con 0x7f35dc0d1f60 2024-04-25T01:08:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.746+0000 7f35d37fe640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f35bc067830 0x7f35bc069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.746+0000 7f35d37fe640 1 -- 172.21.15.33:0/828644960 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f35c4085810 con 0x7f35dc0d1f60 2024-04-25T01:08:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.746+0000 7f35daffd640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f35bc067830 0x7f35bc069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.746+0000 7f35daffd640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f35bc067830 0x7f35bc069cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0af000 tx=0x7f35cc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.750+0000 7f35d37fe640 1 -- 172.21.15.33:0/828644960 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f35c40568d0 con 0x7f35dc0d1f60 2024-04-25T01:08:17.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.914+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f35dc0c7510 con 0x7f35dc0d1f60 2024-04-25T01:08:17.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.914+0000 7f35d37fe640 1 -- 172.21.15.33:0/828644960 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) ==== 93+0+10 (secure 0 0 0) 0x7f35c40561d0 con 0x7f35dc0d1f60 2024-04-25T01:08:17.908 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:08:17.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f35bc067830 msgr2=0x7f35bc069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f35bc067830 0x7f35bc069cf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0af000 tx=0x7f35cc0a7000 comp rx=0 tx=0).stop 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc0d1f60 msgr2=0x7f35dc09efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc0d1f60 0x7f35dc09efe0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f35c400e930 tx=0x7f35c400edf0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 shutdown_connections 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f35bc067830 0x7f35bc069cf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f35dc15e0d0 0x7f35dc15fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35dc0d1f60 0x7f35dc09efe0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 --2- 172.21.15.33:0/828644960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35dc0b9a20 0x7f35dc15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 >> 172.21.15.33:0/828644960 conn(0x7f35dc0d3dd0 msgr2=0x7f35dc0b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 shutdown_connections 2024-04-25T01:08:17.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:17.918+0000 7f35e1470640 1 -- 172.21.15.33:0/828644960 wait complete. 2024-04-25T01:08:17.927 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:18.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cd0926640 1 Processor -- start 2024-04-25T01:08:18.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cd0926640 1 -- start start 2024-04-25T01:08:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cd0926640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d03e0 0x7f1ccc0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cd0926640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ccc15e470 0x7f1ccc0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cd0926640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ccc0d10b0 con 0x7f1ccc0d48d0 2024-04-25T01:08:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cd0926640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1ccc0d1de0 con 0x7f1ccc15e470 2024-04-25T01:08:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cd0926640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc0d1390 con 0x7f1ccc0d03e0 2024-04-25T01:08:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cc3fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ccc15e470 0x7f1ccc0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:18.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cc3fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1ccc15e470 0x7f1ccc0d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51756/0 (socket says 172.21.15.33:51756) 2024-04-25T01:08:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cc3fff640 1 -- 172.21.15.33:0/167404979 learned_addr learned my addr 172.21.15.33:0/167404979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cca59c640 1 --2- 172.21.15.33:0/167404979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d03e0 0x7f1ccc0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cc3fff640 1 -- 172.21.15.33:0/167404979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d03e0 msgr2=0x7f1ccc0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cc3fff640 1 --2- 172.21.15.33:0/167404979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d03e0 0x7f1ccc0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cc3fff640 1 -- 172.21.15.33:0/167404979 >> v1:172.21.15.138:6789/0 conn(0x7f1ccc0d48d0 legacy=0x7f1ccc0d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:18.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.062+0000 7f1cc3fff640 1 -- 172.21.15.33:0/167404979 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc0d1670 con 0x7f1ccc15e470 2024-04-25T01:08:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cc3fff640 1 --2- 172.21.15.33:0/167404979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc0d2270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009870 tx=0x7f1ccc0d36d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb6ae1865e7ae6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cc9d9b640 1 -- 172.21.15.33:0/167404979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40b0070 con 0x7f1ccc15e470 2024-04-25T01:08:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cc9d9b640 1 -- 172.21.15.33:0/167404979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cb40ab600 con 0x7f1ccc15e470 2024-04-25T01:08:18.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cc9d9b640 1 -- 172.21.15.33:0/167404979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40ab9d0 con 0x7f1ccc15e470 2024-04-25T01:08:18.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- 172.21.15.33:0/167404979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 msgr2=0x7f1ccc0d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:18.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 --2- 172.21.15.33:0/167404979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc0d2270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009870 tx=0x7f1ccc0d36d0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- 172.21.15.33:0/167404979 shutdown_connections 2024-04-25T01:08:18.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 --2- 172.21.15.33:0/167404979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc0d2270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 --2- 172.21.15.33:0/167404979 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d03e0 0x7f1ccc0d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- 172.21.15.33:0/167404979 >> 172.21.15.33:0/167404979 conn(0x7f1ccc0d3db0 msgr2=0x7f1ccc0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:18.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- 172.21.15.33:0/167404979 shutdown_connections 2024-04-25T01:08:18.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- 172.21.15.33:0/167404979 wait complete. 2024-04-25T01:08:18.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 Processor -- start 2024-04-25T01:08:18.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- start start 2024-04-25T01:08:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc0d03e0 0x7f1ccc15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d48d0 0x7f1ccc1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:18.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc16af40 con 0x7f1ccc15e470 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc16adc0 con 0x7f1ccc0d03e0 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.066+0000 7f1cd0926640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1ccc16b0c0 con 0x7f1ccc0d48d0 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51770/0 (socket says 172.21.15.33:51770) 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 -- 172.21.15.33:0/2958951360 learned_addr learned my addr 172.21.15.33:0/2958951360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cca59c640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc0d03e0 0x7f1ccc15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc3fff640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d48d0 0x7f1ccc1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 -- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d48d0 msgr2=0x7f1ccc1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d48d0 0x7f1ccc1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 -- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc0d03e0 msgr2=0x7f1ccc15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:18.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc0d03e0 0x7f1ccc15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 -- 172.21.15.33:0/2958951360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc153790 con 0x7f1ccc15e470 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc3fff640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d48d0 0x7f1ccc1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1ccad9d640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc160b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400c770 tx=0x7f1cc400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc2ffd640 1 -- 172.21.15.33:0/2958951360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4003b80 con 0x7f1ccc15e470 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc2ffd640 1 -- 172.21.15.33:0/2958951360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc4003d20 con 0x7f1ccc15e470 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc153a30 con 0x7f1ccc15e470 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc2ffd640 1 -- 172.21.15.33:0/2958951360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400aa90 con 0x7f1ccc15e470 2024-04-25T01:08:18.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1528f0 con 0x7f1ccc15e470 2024-04-25T01:08:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ccc0b7d10 con 0x7f1ccc15e470 2024-04-25T01:08:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc2ffd640 1 -- 172.21.15.33:0/2958951360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f1cc400ac30 con 0x7f1ccc15e470 2024-04-25T01:08:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc2ffd640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cac067860 0x7f1cac069d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cc2ffd640 1 -- 172.21.15.33:0/2958951360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4857+0+0 (secure 0 0 0) 0x7f1cc40856c0 con 0x7f1ccc15e470 2024-04-25T01:08:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cca59c640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cac067860 0x7f1cac069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:18.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.070+0000 7f1cca59c640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cac067860 0x7f1cac069d20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00b9d0 tx=0x7f1cbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:18.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.074+0000 7f1cc2ffd640 1 -- 172.21.15.33:0/2958951360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1cc40566c0 con 0x7f1ccc15e470 2024-04-25T01:08:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.274+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ccc069040 con 0x7f1ccc15e470 2024-04-25T01:08:18.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cc2ffd640 1 -- 172.21.15.33:0/2958951360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f1cc4055fc0 con 0x7f1ccc15e470 2024-04-25T01:08:18.268 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:18.269 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T01:08:18.269 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cac067860 msgr2=0x7f1cac069d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cac067860 0x7f1cac069d20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc00b9d0 tx=0x7f1cbc008040 comp rx=0 tx=0).stop 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 msgr2=0x7f1ccc160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc160b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400c770 tx=0x7f1cc400cc30 comp rx=0 tx=0).stop 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 shutdown_connections 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cac067860 0x7f1cac069d20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc15e470 0x7f1ccc160b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1ccc0d48d0 0x7f1ccc1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 --2- 172.21.15.33:0/2958951360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ccc0d03e0 0x7f1ccc15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 >> 172.21.15.33:0/2958951360 conn(0x7f1ccc0d3db0 msgr2=0x7f1ccc15a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 shutdown_connections 2024-04-25T01:08:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:18.278+0000 7f1cd0926640 1 -- 172.21.15.33:0/2958951360 wait complete. 2024-04-25T01:08:20.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:20.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.426+0000 7f4146ab1640 1 Processor -- start 2024-04-25T01:08:20.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.426+0000 7f4146ab1640 1 -- start start 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f4146ab1640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41401444f0 0x7f41400b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f4146ab1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4140072540 0x7f41400ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f4146ab1640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41401448d0 con 0x7f414015e660 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f4146ab1640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f41400ce0c0 con 0x7f41401444f0 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f4146ab1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f41400b5b60 con 0x7f4140072540 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414484d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41401444f0 0x7f41400b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414484d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f41401444f0 0x7f41400b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44520/0 (socket says 172.21.15.33:44520) 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414484d640 1 -- 172.21.15.33:0/780346612 learned_addr learned my addr 172.21.15.33:0/780346612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414504e640 1 --2- 172.21.15.33:0/780346612 >> v2:172.21.15.138:3300/0 conn(0x7f4140072540 0x7f41400ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414484d640 1 -- 172.21.15.33:0/780346612 >> v2:172.21.15.138:3300/0 conn(0x7f4140072540 msgr2=0x7f41400ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:20.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414484d640 1 --2- 172.21.15.33:0/780346612 >> v2:172.21.15.138:3300/0 conn(0x7f4140072540 0x7f41400ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414484d640 1 -- 172.21.15.33:0/780346612 >> v1:172.21.15.33:6789/0 conn(0x7f414015e660 legacy=0x7f4140071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.430+0000 7f414484d640 1 -- 172.21.15.33:0/780346612 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414015ec20 con 0x7f41401444f0 2024-04-25T01:08:20.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f414484d640 1 --2- 172.21.15.33:0/780346612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41401444f0 0x7f41400b5620 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41400b2ba0 tx=0x7f413001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f504ca78474ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f413f7fe640 1 -- 172.21.15.33:0/780346612 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41300b0070 con 0x7f41401444f0 2024-04-25T01:08:20.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f413f7fe640 1 -- 172.21.15.33:0/780346612 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41300ab600 con 0x7f41401444f0 2024-04-25T01:08:20.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f413f7fe640 1 -- 172.21.15.33:0/780346612 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41300ab9d0 con 0x7f41401444f0 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 -- 172.21.15.33:0/780346612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41401444f0 msgr2=0x7f41400b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 --2- 172.21.15.33:0/780346612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41401444f0 0x7f41400b5620 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f41400b2ba0 tx=0x7f413001c490 comp rx=0 tx=0).stop 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 -- 172.21.15.33:0/780346612 shutdown_connections 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 --2- 172.21.15.33:0/780346612 >> v2:172.21.15.138:3300/0 conn(0x7f4140072540 0x7f41400ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 --2- 172.21.15.33:0/780346612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41401444f0 0x7f41400b5620 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 -- 172.21.15.33:0/780346612 >> 172.21.15.33:0/780346612 conn(0x7f41400b1810 msgr2=0x7f41400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 -- 172.21.15.33:0/780346612 shutdown_connections 2024-04-25T01:08:20.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 -- 172.21.15.33:0/780346612 wait complete. 2024-04-25T01:08:20.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 Processor -- start 2024-04-25T01:08:20.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.434+0000 7f4146ab1640 1 -- start start 2024-04-25T01:08:20.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140072540 0x7f4140156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41401444f0 0x7f414015b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414015e660 0x7f414015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41400b37a0 con 0x7f4140072540 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f41400b3620 con 0x7f414015e660 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4140070d20 con 0x7f41401444f0 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140072540 0x7f4140156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140072540 0x7f4140156bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44524/0 (socket says 172.21.15.33:44524) 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 -- 172.21.15.33:0/4069106406 learned_addr learned my addr 172.21.15.33:0/4069106406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414504e640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414015e660 0x7f414015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 -- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41401444f0 msgr2=0x7f414015b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41401444f0 0x7f414015b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 -- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414015e660 msgr2=0x7f414015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414015e660 0x7f414015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 -- 172.21.15.33:0/4069106406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41401314c0 con 0x7f4140072540 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f414484d640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140072540 0x7f4140156bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41300ae040 tx=0x7f4130019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f413d7fa640 1 -- 172.21.15.33:0/4069106406 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41300b0040 con 0x7f4140072540 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f413d7fa640 1 -- 172.21.15.33:0/4069106406 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41300b9070 con 0x7f4140072540 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4140138fe0 con 0x7f4140072540 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f413d7fa640 1 -- 172.21.15.33:0/4069106406 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41300b4680 con 0x7f4140072540 2024-04-25T01:08:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414015a950 con 0x7f4140072540 2024-04-25T01:08:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4140069040 con 0x7f4140072540 2024-04-25T01:08:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f413d7fa640 1 -- 172.21.15.33:0/4069106406 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f41300b4820 con 0x7f4140072540 2024-04-25T01:08:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f413d7fa640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f412c067830 0x7f412c069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f413d7fa640 1 -- 172.21.15.33:0/4069106406 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f413011f360 con 0x7f4140072540 2024-04-25T01:08:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.438+0000 7f413ffff640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f412c067830 0x7f412c069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.442+0000 7f413ffff640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f412c067830 0x7f412c069cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4134007fd0 tx=0x7f4134007d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.446+0000 7f413d7fa640 1 -- 172.21.15.33:0/4069106406 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4130016110 con 0x7f4140072540 2024-04-25T01:08:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.646+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41400b2f20 con 0x7f4140072540 2024-04-25T01:08:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.646+0000 7f413d7fa640 1 -- 172.21.15.33:0/4069106406 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3867 (secure 0 0 0) 0x7f41300f0360 con 0x7f4140072540 2024-04-25T01:08:20.639 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:20.639 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4248,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T01:08:20.639 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 3 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f412c067830 msgr2=0x7f412c069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f412c067830 0x7f412c069cf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4134007fd0 tx=0x7f4134007d40 comp rx=0 tx=0).stop 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140072540 msgr2=0x7f4140156bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140072540 0x7f4140156bf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41300ae040 tx=0x7f4130019230 comp rx=0 tx=0).stop 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 shutdown_connections 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f412c067830 0x7f412c069cf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f414015e660 0x7f414015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f41401444f0 0x7f414015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 --2- 172.21.15.33:0/4069106406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140072540 0x7f4140156bf0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f41300ae040 tx=0x7f4130019230 comp rx=0 tx=0).stop 2024-04-25T01:08:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 >> 172.21.15.33:0/4069106406 conn(0x7f41400b1810 msgr2=0x7f41400b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 shutdown_connections 2024-04-25T01:08:20.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.650+0000 7f4146ab1640 1 -- 172.21.15.33:0/4069106406 wait complete. 2024-04-25T01:08:20.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.782+0000 7f27c407f640 1 Processor -- start 2024-04-25T01:08:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.782+0000 7f27c407f640 1 -- start start 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27bc15c160 0x7f27bc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f27bc0b5360 0x7f27bc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc15f9a0 con 0x7f27bc145470 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f27bc0b57b0 con 0x7f27bc15c160 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f27bc15f750 con 0x7f27bc0b5360 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27bc15c160 0x7f27bc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f27bc15c160 0x7f27bc0b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44534/0 (socket says 172.21.15.33:44534) 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 -- 172.21.15.33:0/3429833853 learned_addr learned my addr 172.21.15.33:0/3429833853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c261c640 1 --2- 172.21.15.33:0/3429833853 >> v2:172.21.15.138:3300/0 conn(0x7f27bc0b5360 0x7f27bc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 -- 172.21.15.33:0/3429833853 >> v2:172.21.15.138:3300/0 conn(0x7f27bc0b5360 msgr2=0x7f27bc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 --2- 172.21.15.33:0/3429833853 >> v2:172.21.15.138:3300/0 conn(0x7f27bc0b5360 0x7f27bc0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 -- 172.21.15.33:0/3429833853 >> v1:172.21.15.138:6789/0 conn(0x7f27bc145470 legacy=0x7f27bc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:20.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 -- 172.21.15.33:0/3429833853 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc15fc80 con 0x7f27bc15c160 2024-04-25T01:08:20.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c261c640 1 --2- 172.21.15.33:0/3429833853 >> v2:172.21.15.138:3300/0 conn(0x7f27bc0b5360 0x7f27bc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c161a640 1 --2- 172.21.15.33:0/3429833853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc15c160 0x7f27bc0b4e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0713f0 tx=0x7f27ac01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e23ef5f99f6364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c0e19640 1 -- 172.21.15.33:0/3429833853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0a6040 con 0x7f27bc15c160 2024-04-25T01:08:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c0e19640 1 -- 172.21.15.33:0/3429833853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27ac0af070 con 0x7f27bc15c160 2024-04-25T01:08:20.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c0e19640 1 -- 172.21.15.33:0/3429833853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0aa860 con 0x7f27bc15c160 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- 172.21.15.33:0/3429833853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc15c160 msgr2=0x7f27bc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 --2- 172.21.15.33:0/3429833853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc15c160 0x7f27bc0b4e20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f27bc0713f0 tx=0x7f27ac01c4e0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- 172.21.15.33:0/3429833853 shutdown_connections 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 --2- 172.21.15.33:0/3429833853 >> v2:172.21.15.138:3300/0 conn(0x7f27bc0b5360 0x7f27bc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 --2- 172.21.15.33:0/3429833853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc15c160 0x7f27bc0b4e20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- 172.21.15.33:0/3429833853 >> 172.21.15.33:0/3429833853 conn(0x7f27bc0b1810 msgr2=0x7f27bc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- 172.21.15.33:0/3429833853 shutdown_connections 2024-04-25T01:08:20.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.786+0000 7f27c407f640 1 -- 172.21.15.33:0/3429833853 wait complete. 2024-04-25T01:08:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 Processor -- start 2024-04-25T01:08:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 -- start start 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc0b5360 0x7f27bc0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27bc145470 0x7f27bc143490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27bc15c160 0x7f27bc15ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc06e850 con 0x7f27bc0b5360 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc145850 con 0x7f27bc145470 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f27bc168af0 con 0x7f27bc15c160 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc0b5360 0x7f27bc0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc0b5360 0x7f27bc0b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44546/0 (socket says 172.21.15.33:44546) 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 -- 172.21.15.33:0/3278465907 learned_addr learned my addr 172.21.15.33:0/3278465907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:20.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c161a640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27bc145470 0x7f27bc143490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c261c640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27bc15c160 0x7f27bc15ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 -- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27bc15c160 msgr2=0x7f27bc15ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27bc15c160 0x7f27bc15ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 -- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27bc145470 msgr2=0x7f27bc143490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27bc145470 0x7f27bc143490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 -- 172.21.15.33:0/3278465907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc143d00 con 0x7f27bc0b5360 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c161a640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27bc145470 0x7f27bc143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c1e1b640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc0b5360 0x7f27bc0b41d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c450 tx=0x7f27b800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27b27fc640 1 -- 172.21.15.33:0/3278465907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8012830 con 0x7f27bc0b5360 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27b27fc640 1 -- 172.21.15.33:0/3278465907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27b8017070 con 0x7f27bc0b5360 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27b27fc640 1 -- 172.21.15.33:0/3278465907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b8012c30 con 0x7f27bc0b5360 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc143fe0 con 0x7f27bc0b5360 2024-04-25T01:08:20.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1445a0 con 0x7f27bc0b5360 2024-04-25T01:08:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.790+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27bc069040 con 0x7f27bc0b5360 2024-04-25T01:08:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.794+0000 7f27b27fc640 1 -- 172.21.15.33:0/3278465907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f27b801c460 con 0x7f27bc0b5360 2024-04-25T01:08:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.794+0000 7f27b27fc640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f27a0067830 0x7f27a0069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:20.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.794+0000 7f27b27fc640 1 -- 172.21.15.33:0/3278465907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f27b80856b0 con 0x7f27bc0b5360 2024-04-25T01:08:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.794+0000 7f27c161a640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f27a0067830 0x7f27a0069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:20.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.794+0000 7f27c161a640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f27a0067830 0x7f27a0069cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0ae000 tx=0x7f27ac0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:20.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.798+0000 7f27b27fc640 1 -- 172.21.15.33:0/3278465907 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f27b8056590 con 0x7f27bc0b5360 2024-04-25T01:08:20.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.962+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27bc071420 con 0x7f27bc0b5360 2024-04-25T01:08:20.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.962+0000 7f27b27fc640 1 -- 172.21.15.33:0/3278465907 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12873 (secure 0 0 0) 0x7f27b8055e90 con 0x7f27bc0b5360 2024-04-25T01:08:20.956 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:20.956 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:19.368701+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f27a0067830 msgr2=0x7f27a0069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f27a0067830 0x7f27a0069cf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0ae000 tx=0x7f27ac0a7000 comp rx=0 tx=0).stop 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc0b5360 msgr2=0x7f27bc0b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc0b5360 0x7f27bc0b41d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f27b800c450 tx=0x7f27b800c910 comp rx=0 tx=0).stop 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 shutdown_connections 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f27a0067830 0x7f27a0069cf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f27bc15c160 0x7f27bc15ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f27bc145470 0x7f27bc143490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 --2- 172.21.15.33:0/3278465907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27bc0b5360 0x7f27bc0b41d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 >> 172.21.15.33:0/3278465907 conn(0x7f27bc0b1810 msgr2=0x7f27bc0cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 shutdown_connections 2024-04-25T01:08:20.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:20.966+0000 7f27c407f640 1 -- 172.21.15.33:0/3278465907 wait complete. 2024-04-25T01:08:20.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:08:20.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:08:21.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc780f27640 1 Processor -- start 2024-04-25T01:08:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc780f27640 1 -- start start 2024-04-25T01:08:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc780f27640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc77c0aedf0 0x7fc77c168f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc780f27640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c15e480 con 0x7fc77c15e0a0 2024-04-25T01:08:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc780f27640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc77c06df00 con 0x7fc77c0716a0 2024-04-25T01:08:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc780f27640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc77c071b30 con 0x7fc77c0aedf0 2024-04-25T01:08:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc77ad9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc77c0aedf0 0x7fc77c168f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc77a59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc77c15e0a0 0x7fc77c0b31a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33704/0 (socket says 172.21.15.33:33704) 2024-04-25T01:08:21.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc77a59c640 1 -- 172.21.15.33:0/1949064101 learned_addr learned my addr 172.21.15.33:0/1949064101 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc77ad9d640 1 -- 172.21.15.33:0/1949064101 >> v1:172.21.15.138:6789/0 conn(0x7fc77c0716a0 legacy=0x7fc77c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc77959a640 1 -- 172.21.15.33:0/1949064101 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374653038 0 0) 0x7fc77c15e480 con 0x7fc77c15e0a0 2024-04-25T01:08:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc77ad9d640 1 -- 172.21.15.33:0/1949064101 >> v1:172.21.15.33:6789/0 conn(0x7fc77c15e0a0 legacy=0x7fc77c0b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.110+0000 7fc77ad9d640 1 -- 172.21.15.33:0/1949064101 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c0ae220 con 0x7fc77c0aedf0 2024-04-25T01:08:21.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc77ad9d640 1 --2- 172.21.15.33:0/1949064101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c168f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7740097c0 tx=0x7fc77401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63573a06d9e9c785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc77959a640 1 -- 172.21.15.33:0/1949064101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740b0070 con 0x7fc77c0aedf0 2024-04-25T01:08:21.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc77959a640 1 -- 172.21.15.33:0/1949064101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7740ab600 con 0x7fc77c0aedf0 2024-04-25T01:08:21.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc77959a640 1 -- 172.21.15.33:0/1949064101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740ab9d0 con 0x7fc77c0aedf0 2024-04-25T01:08:21.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- 172.21.15.33:0/1949064101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 msgr2=0x7fc77c168f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 --2- 172.21.15.33:0/1949064101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c168f60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc7740097c0 tx=0x7fc77401c410 comp rx=0 tx=0).stop 2024-04-25T01:08:21.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- 172.21.15.33:0/1949064101 shutdown_connections 2024-04-25T01:08:21.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 --2- 172.21.15.33:0/1949064101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c168f60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- 172.21.15.33:0/1949064101 >> 172.21.15.33:0/1949064101 conn(0x7fc77c0b1810 msgr2=0x7fc77c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:21.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- 172.21.15.33:0/1949064101 shutdown_connections 2024-04-25T01:08:21.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- 172.21.15.33:0/1949064101 wait complete. 2024-04-25T01:08:21.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 Processor -- start 2024-04-25T01:08:21.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- start start 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c0716a0 0x7fc77c14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c14ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc77c15e0a0 0x7fc77c157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c16be40 con 0x7fc77c0aedf0 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc77c16bcc0 con 0x7fc77c0716a0 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc77c16bfc0 con 0x7fc77c15e0a0 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c14ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c14ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44560/0 (socket says 172.21.15.33:44560) 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 -- 172.21.15.33:0/3132801905 learned_addr learned my addr 172.21.15.33:0/3132801905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 -- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc77c15e0a0 msgr2=0x7fc77c157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc77c15e0a0 0x7fc77c157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc77a59c640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c0716a0 0x7fc77c14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 -- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c0716a0 msgr2=0x7fc77c14d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c0716a0 0x7fc77c14d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 -- 172.21.15.33:0/3132801905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77c155a90 con 0x7fc77c0aedf0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc779d9b640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c14ad70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c3a0 tx=0x7fc77000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc76affd640 1 -- 172.21.15.33:0/3132801905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7700052d0 con 0x7fc77c0aedf0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc76affd640 1 -- 172.21.15.33:0/3132801905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc770005470 con 0x7fc77c0aedf0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77c155d70 con 0x7fc77c0aedf0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.114+0000 7fc76affd640 1 -- 172.21.15.33:0/3132801905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770007260 con 0x7fc77c0aedf0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.118+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc77c156c50 con 0x7fc77c0aedf0 2024-04-25T01:08:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.118+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77c068c60 con 0x7fc77c0aedf0 2024-04-25T01:08:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.118+0000 7fc76affd640 1 -- 172.21.15.33:0/3132801905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7fc770007400 con 0x7fc77c0aedf0 2024-04-25T01:08:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.118+0000 7fc76affd640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc75c067830 0x7fc75c069cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.118+0000 7fc76affd640 1 -- 172.21.15.33:0/3132801905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fc770084ca0 con 0x7fc77c0aedf0 2024-04-25T01:08:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.118+0000 7fc77a59c640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc75c067830 0x7fc75c069cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.118+0000 7fc77a59c640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc75c067830 0x7fc75c069cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0012f0 tx=0x7fc76c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.122+0000 7fc76affd640 1 -- 172.21.15.33:0/3132801905 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc770055c10 con 0x7fc77c0aedf0 2024-04-25T01:08:21.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.282+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc77c069040 con 0x7fc77c0aedf0 2024-04-25T01:08:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc76affd640 1 -- 172.21.15.33:0/3132801905 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fc770055510 con 0x7fc77c0aedf0 2024-04-25T01:08:21.277 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:08:21.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc75c067830 msgr2=0x7fc75c069cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc75c067830 0x7fc75c069cf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc76c0012f0 tx=0x7fc76c009040 comp rx=0 tx=0).stop 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 msgr2=0x7fc77c14ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c14ad70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c3a0 tx=0x7fc77000c860 comp rx=0 tx=0).stop 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 shutdown_connections 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc75c067830 0x7fc75c069cf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc77c15e0a0 0x7fc77c157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c0aedf0 0x7fc77c14ad70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 --2- 172.21.15.33:0/3132801905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c0716a0 0x7fc77c14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 >> 172.21.15.33:0/3132801905 conn(0x7fc77c0b1810 msgr2=0x7fc77c0cbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 shutdown_connections 2024-04-25T01:08:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.286+0000 7fc780f27640 1 -- 172.21.15.33:0/3132801905 wait complete. 2024-04-25T01:08:21.296 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T01:08:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8999c68640 1 Processor -- start 2024-04-25T01:08:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8999c68640 1 -- start start 2024-04-25T01:08:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8999c68640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f899415c160 0x7f89940b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8999c68640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f89940cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8999c68640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f899415f9a0 con 0x7f8994145470 2024-04-25T01:08:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8999c68640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f89940b57b0 con 0x7f899415c160 2024-04-25T01:08:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8999c68640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f899415f750 con 0x7f89940b5360 2024-04-25T01:08:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8992ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f899415c160 0x7f89940b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8992ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f899415c160 0x7f89940b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:44446/0 (socket says 172.21.15.33:44446) 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8992ffd640 1 -- 172.21.15.33:0/1248778181 learned_addr learned my addr 172.21.15.33:0/1248778181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f89937fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8994145470 0x7f89940cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:57438/0 (socket says 172.21.15.33:57438) 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8993fff640 1 --2- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f89940cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8992ffd640 1 -- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 msgr2=0x7f89940cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8992ffd640 1 --2- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f89940cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8992ffd640 1 -- 172.21.15.33:0/1248778181 >> v1:172.21.15.138:6789/0 conn(0x7f8994145470 legacy=0x7f89940cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.434+0000 7f8992ffd640 1 -- 172.21.15.33:0/1248778181 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899415fc80 con 0x7f899415c160 2024-04-25T01:08:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8992ffd640 1 --2- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f899415c160 0x7f89940b4e20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8988009870 tx=0x7f898801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f805ef9cbb5c43b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f89927fc640 1 -- 172.21.15.33:0/1248778181 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880a7070 con 0x7f899415c160 2024-04-25T01:08:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f89927fc640 1 -- 172.21.15.33:0/1248778181 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89880b0070 con 0x7f899415c160 2024-04-25T01:08:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f89927fc640 1 -- 172.21.15.33:0/1248778181 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89880ab890 con 0x7f899415c160 2024-04-25T01:08:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f899415c160 msgr2=0x7f89940b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 --2- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f899415c160 0x7f89940b4e20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8988009870 tx=0x7f898801c560 comp rx=0 tx=0).stop 2024-04-25T01:08:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- 172.21.15.33:0/1248778181 shutdown_connections 2024-04-25T01:08:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 --2- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f89940cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 --2- 172.21.15.33:0/1248778181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f899415c160 0x7f89940b4e20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- 172.21.15.33:0/1248778181 >> 172.21.15.33:0/1248778181 conn(0x7f89940b1810 msgr2=0x7f89940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- 172.21.15.33:0/1248778181 shutdown_connections 2024-04-25T01:08:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- 172.21.15.33:0/1248778181 wait complete. 2024-04-25T01:08:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 Processor -- start 2024-04-25T01:08:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- start start 2024-04-25T01:08:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f8994137ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8994145470 0x7f899415ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f899415c160 0x7f899415e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f899406e850 con 0x7f899415c160 2024-04-25T01:08:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8994145850 con 0x7f8994145470 2024-04-25T01:08:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.438+0000 7f8999c68640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8994166ae0 con 0x7f89940b5360 2024-04-25T01:08:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f899415c160 0x7f899415e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f899415c160 0x7f899415e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44566/0 (socket says 172.21.15.33:44566) 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 -- 172.21.15.33:0/2681303632 learned_addr learned my addr 172.21.15.33:0/2681303632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f89937fe640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f8994137ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8992ffd640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8994145470 0x7f899415ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 -- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 msgr2=0x7f8994137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f8994137ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 -- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8994145470 msgr2=0x7f899415ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8994145470 0x7f899415ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 -- 172.21.15.33:0/2681303632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f899415eb60 con 0x7f899415c160 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8993fff640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f899415c160 0x7f899415e260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f898c00efc0 tx=0x7f898c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f897bfff640 1 -- 172.21.15.33:0/2681303632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c018020 con 0x7f899415c160 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f897bfff640 1 -- 172.21.15.33:0/2681303632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f898c014480 con 0x7f899415c160 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8994134300 con 0x7f899415c160 2024-04-25T01:08:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f897bfff640 1 -- 172.21.15.33:0/2681303632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c014870 con 0x7f899415c160 2024-04-25T01:08:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f899415b2a0 con 0x7f899415c160 2024-04-25T01:08:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8994069040 con 0x7f899415c160 2024-04-25T01:08:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f897bfff640 1 -- 172.21.15.33:0/2681303632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f898c01b020 con 0x7f899415c160 2024-04-25T01:08:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f897bfff640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f89800678c0 0x7f8980069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f897bfff640 1 -- 172.21.15.33:0/2681303632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f898c086730 con 0x7f899415c160 2024-04-25T01:08:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.442+0000 7f89937fe640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f89800678c0 0x7f8980069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.446+0000 7f89937fe640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f89800678c0 0x7f8980069d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8984000f70 tx=0x7f8984007c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.446+0000 7f897bfff640 1 -- 172.21.15.33:0/2681303632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f898c022030 con 0x7f899415c160 2024-04-25T01:08:21.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.614+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8994071420 con 0x7f899415c160 2024-04-25T01:08:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.614+0000 7f897bfff640 1 -- 172.21.15.33:0/2681303632 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f898c010040 con 0x7f899415c160 2024-04-25T01:08:21.606 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:21.606 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-04-25T01:08:21.607 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:08:21.607 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-04-25T01:08:21.607 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-04-25T01:08:21.607 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:08:21.607 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f89800678c0 msgr2=0x7f8980069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f89800678c0 0x7f8980069d80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8984000f70 tx=0x7f8984007c50 comp rx=0 tx=0).stop 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f899415c160 msgr2=0x7f899415e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f899415c160 0x7f899415e260 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f898c00efc0 tx=0x7f898c00c490 comp rx=0 tx=0).stop 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 shutdown_connections 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f89800678c0 0x7f8980069d80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f899415c160 0x7f899415e260 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8994145470 0x7f899415ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 --2- 172.21.15.33:0/2681303632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f89940b5360 0x7f8994137ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 >> 172.21.15.33:0/2681303632 conn(0x7f89940b1810 msgr2=0x7f899406d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:21.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 shutdown_connections 2024-04-25T01:08:21.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.618+0000 7f8999c68640 1 -- 172.21.15.33:0/2681303632 wait complete. 2024-04-25T01:08:21.626 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T01:08:21.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T01:08:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb4240640 1 Processor -- start 2024-04-25T01:08:21.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb4240640 1 -- start start 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb4240640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbeac144860 0x7fbeac0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb4240640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 0x7fbeac06b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb4240640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeac0ceef0 con 0x7fbeac070460 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb4240640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbeac0708a0 con 0x7fbeac144860 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb4240640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac0cf1d0 con 0x7fbeac06b210 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb17db640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbeac070460 0x7fbeac0ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33718/0 (socket says 172.21.15.33:33718) 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb17db640 1 -- 172.21.15.33:0/2070599248 learned_addr learned my addr 172.21.15.33:0/2070599248 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445920948 0 0) 0x7fbeac0ceef0 con 0x7fbeac070460 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb27dd640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 0x7fbeac06b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb1fdc640 1 --2- 172.21.15.33:0/2070599248 >> v2:172.21.15.138:3300/0 conn(0x7fbeac144860 0x7fbeac0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe98003530 con 0x7fbeac070460 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2946857218 0 0) 0x7fbe98003530 con 0x7fbeac070460 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 msgr2=0x7fbeac06b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb0fda640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 0x7fbeac06b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 >> v2:172.21.15.138:3300/0 conn(0x7fbeac144860 msgr2=0x7fbeac0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.758+0000 7fbeb0fda640 1 --2- 172.21.15.33:0/2070599248 >> v2:172.21.15.138:3300/0 conn(0x7fbeac144860 0x7fbeac0cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac0cf4b0 con 0x7fbeac070460 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fbe9c001680 con 0x7fbeac070460 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 >> v1:172.21.15.33:6789/0 conn(0x7fbeac070460 legacy=0x7fbeac0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe980039c0 0x7fbe98003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe98004810 0x7fbe98004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 0x7fbe980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0cf1d0 con 0x7fbe98004810 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9800b9d0 con 0x7fbe980039c0 2024-04-25T01:08:21.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbe9800bb50 con 0x7fbeac06b210 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb17db640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe98004810 0x7fbe98004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb1fdc640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe980039c0 0x7fbe98003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb27dd640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 0x7fbe980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb17db640 1 -- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 msgr2=0x7fbe980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb17db640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac06b210 0x7fbe980051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb17db640 1 -- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe980039c0 msgr2=0x7fbe98003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb17db640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe980039c0 0x7fbe98003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb17db640 1 -- 172.21.15.33:0/2070599248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe98003530 con 0x7fbe98004810 2024-04-25T01:08:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb17db640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe98004810 0x7fbe98004c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c001060 tx=0x7fbe9c01f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5656e1bd6acef5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c007d80 con 0x7fbe98004810 2024-04-25T01:08:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe9c005b00 con 0x7fbe98004810 2024-04-25T01:08:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb0fda640 1 -- 172.21.15.33:0/2070599248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c007070 con 0x7fbe98004810 2024-04-25T01:08:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb4240640 1 -- 172.21.15.33:0/2070599248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe98004810 msgr2=0x7fbe98004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb4240640 1 --2- 172.21.15.33:0/2070599248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe98004810 0x7fbe98004c90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c001060 tx=0x7fbe9c01f410 comp rx=0 tx=0).stop 2024-04-25T01:08:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb27dd640 1 -- 172.21.15.33:0/2070599248 reap_dead start 2024-04-25T01:08:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb4240640 1 -- 172.21.15.33:0/2070599248 shutdown_connections 2024-04-25T01:08:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb4240640 1 -- 172.21.15.33:0/2070599248 >> 172.21.15.33:0/2070599248 conn(0x7fbeac0d0af0 msgr2=0x7fbeac0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb4240640 1 -- 172.21.15.33:0/2070599248 shutdown_connections 2024-04-25T01:08:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.762+0000 7fbeb4240640 1 -- 172.21.15.33:0/2070599248 wait complete. 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 Processor -- start 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 -- start start 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98004810 0x7fbeac13ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac145cc0 0x7fbeac13d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac13d780 0x7fbeac13dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac06b6a0 con 0x7fbeac13d780 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0cfa70 con 0x7fbe98004810 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac06b820 con 0x7fbeac145cc0 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac13d780 0x7fbeac13dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac13d780 0x7fbeac13dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44574/0 (socket says 172.21.15.33:44574) 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 -- 172.21.15.33:0/2884841485 learned_addr learned my addr 172.21.15.33:0/2884841485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb1fdc640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98004810 0x7fbeac13ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb17db640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac145cc0 0x7fbeac13d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 -- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac145cc0 msgr2=0x7fbeac13d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac145cc0 0x7fbeac13d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 -- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98004810 msgr2=0x7fbeac13ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98004810 0x7fbeac13ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 -- 172.21.15.33:0/2884841485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac13e140 con 0x7fbeac13d780 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb17db640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac145cc0 0x7fbeac13d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb27dd640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac13d780 0x7fbeac13dc00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbeac161ab0 tx=0x7fbea400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbea27fc640 1 -- 172.21.15.33:0/2884841485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea401c070 con 0x7fbeac13d780 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbea27fc640 1 -- 172.21.15.33:0/2884841485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbea4017400 con 0x7fbeac13d780 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac146380 con 0x7fbeac13d780 2024-04-25T01:08:21.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbea27fc640 1 -- 172.21.15.33:0/2884841485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea40177f0 con 0x7fbeac13d780 2024-04-25T01:08:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac146940 con 0x7fbeac13d780 2024-04-25T01:08:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.766+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac069040 con 0x7fbeac13d780 2024-04-25T01:08:21.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.770+0000 7fbea27fc640 1 -- 172.21.15.33:0/2884841485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbea401f070 con 0x7fbeac13d780 2024-04-25T01:08:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.770+0000 7fbea27fc640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbe900678f0 0x7fbe90069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.770+0000 7fbea27fc640 1 -- 172.21.15.33:0/2884841485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fbea4085970 con 0x7fbeac13d780 2024-04-25T01:08:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.770+0000 7fbeb1fdc640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbe900678f0 0x7fbe90069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:21.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.770+0000 7fbeb1fdc640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbe900678f0 0x7fbe90069db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009ae0 tx=0x7fbea8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:21.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.774+0000 7fbea27fc640 1 -- 172.21.15.33:0/2884841485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbea4012070 con 0x7fbeac13d780 2024-04-25T01:08:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:21.934+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbeac0b0020 con 0x7fbeac13d780 2024-04-25T01:08:22.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.418+0000 7fbea27fc640 1 -- 172.21.15.33:0/2884841485 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fbea4056240 con 0x7fbeac13d780 2024-04-25T01:08:22.411 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbe900678f0 msgr2=0x7fbe90069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbe900678f0 0x7fbe90069db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbea8009ae0 tx=0x7fbea8008040 comp rx=0 tx=0).stop 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac13d780 msgr2=0x7fbeac13dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac13d780 0x7fbeac13dc00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbeac161ab0 tx=0x7fbea400f4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 shutdown_connections 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbe900678f0 0x7fbe90069db0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac13d780 0x7fbeac13dc00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbeac145cc0 0x7fbeac13d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 --2- 172.21.15.33:0/2884841485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe98004810 0x7fbeac13ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 >> 172.21.15.33:0/2884841485 conn(0x7fbeac0d0af0 msgr2=0x7fbeac0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:22.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 shutdown_connections 2024-04-25T01:08:22.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.422+0000 7fbeb4240640 1 -- 172.21.15.33:0/2884841485 wait complete. 2024-04-25T01:08:22.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T01:08:22.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faafcf55640 1 Processor -- start 2024-04-25T01:08:22.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faafcf55640 1 -- start start 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faafcf55640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faaf806eb60 0x7faaf8160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faafcf55640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faaf8071170 0x7faaf80715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faafcf55640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faaf80cee10 con 0x7faaf8145660 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faafcf55640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7faaf8145a40 con 0x7faaf806eb60 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faafcf55640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faaf8070ce0 con 0x7faaf8071170 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf659c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faaf8145660 0x7faaf815f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33734/0 (socket says 172.21.15.33:33734) 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf659c640 1 -- 172.21.15.33:0/163031335 learned_addr learned my addr 172.21.15.33:0/163031335 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf5d9b640 1 --2- 172.21.15.33:0/163031335 >> v2:172.21.15.33:3300/0 conn(0x7faaf806eb60 0x7faaf8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf6d9d640 1 --2- 172.21.15.33:0/163031335 >> v2:172.21.15.138:3300/0 conn(0x7faaf8071170 0x7faaf80715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf559a640 1 -- 172.21.15.33:0/163031335 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1800517652 0 0) 0x7faaf80cee10 con 0x7faaf8145660 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf559a640 1 -- 172.21.15.33:0/163031335 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faae4003530 con 0x7faaf8145660 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf5d9b640 1 -- 172.21.15.33:0/163031335 >> v2:172.21.15.138:3300/0 conn(0x7faaf8071170 msgr2=0x7faaf80715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf5d9b640 1 --2- 172.21.15.33:0/163031335 >> v2:172.21.15.138:3300/0 conn(0x7faaf8071170 0x7faaf80715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf5d9b640 1 -- 172.21.15.33:0/163031335 >> v1:172.21.15.33:6789/0 conn(0x7faaf8145660 legacy=0x7faaf815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:22.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.586+0000 7faaf5d9b640 1 -- 172.21.15.33:0/163031335 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf80cf0f0 con 0x7faaf806eb60 2024-04-25T01:08:22.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faaf5d9b640 1 --2- 172.21.15.33:0/163031335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf8160070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faaf8068b80 tx=0x7faae001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddde13b8c65e42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:22.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faaf559a640 1 -- 172.21.15.33:0/163031335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00ab650 con 0x7faaf806eb60 2024-04-25T01:08:22.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faaf559a640 1 -- 172.21.15.33:0/163031335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae00b0070 con 0x7faaf806eb60 2024-04-25T01:08:22.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faaf559a640 1 -- 172.21.15.33:0/163031335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae00aba10 con 0x7faaf806eb60 2024-04-25T01:08:22.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 -- 172.21.15.33:0/163031335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 msgr2=0x7faaf8160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:22.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 --2- 172.21.15.33:0/163031335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf8160070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faaf8068b80 tx=0x7faae001c450 comp rx=0 tx=0).stop 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 -- 172.21.15.33:0/163031335 shutdown_connections 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 --2- 172.21.15.33:0/163031335 >> v2:172.21.15.138:3300/0 conn(0x7faaf8071170 0x7faaf80715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 --2- 172.21.15.33:0/163031335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf8160070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 -- 172.21.15.33:0/163031335 >> 172.21.15.33:0/163031335 conn(0x7faaf80b1810 msgr2=0x7faaf80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 -- 172.21.15.33:0/163031335 shutdown_connections 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.590+0000 7faafcf55640 1 -- 172.21.15.33:0/163031335 wait complete. 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 Processor -- start 2024-04-25T01:08:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 -- start start 2024-04-25T01:08:22.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf81593d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:22.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaf8071170 0x7faaf80aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:22.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8145660 0x7faaf8143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:22.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faaf8160730 con 0x7faaf806eb60 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faaf81605b0 con 0x7faaf8145660 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faaf8168ca0 con 0x7faaf8071170 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf81593d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf81593d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44592/0 (socket says 172.21.15.33:44592) 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 -- 172.21.15.33:0/1931341037 learned_addr learned my addr 172.21.15.33:0/1931341037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf6d9d640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8145660 0x7faaf8143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 -- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaf8071170 msgr2=0x7faaf80aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf5d9b640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaf8071170 0x7faaf80aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaf8071170 0x7faaf80aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 -- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8145660 msgr2=0x7faaf8143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8145660 0x7faaf8143f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 -- 172.21.15.33:0/1931341037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf80adf00 con 0x7faaf806eb60 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaf659c640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf81593d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faae800c3c0 tx=0x7faae800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaeeffd640 1 -- 172.21.15.33:0/1931341037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae8017070 con 0x7faaf806eb60 2024-04-25T01:08:22.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaeeffd640 1 -- 172.21.15.33:0/1931341037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faae8005cf0 con 0x7faaf806eb60 2024-04-25T01:08:22.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf80ae1e0 con 0x7faaf806eb60 2024-04-25T01:08:22.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaeeffd640 1 -- 172.21.15.33:0/1931341037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae80072f0 con 0x7faaf806eb60 2024-04-25T01:08:22.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf80ae7a0 con 0x7faaf806eb60 2024-04-25T01:08:22.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf8069040 con 0x7faaf806eb60 2024-04-25T01:08:22.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.594+0000 7faaeeffd640 1 -- 172.21.15.33:0/1931341037 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faae801a070 con 0x7faaf806eb60 2024-04-25T01:08:22.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.598+0000 7faaeeffd640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faad80678c0 0x7faad8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:22.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.598+0000 7faaeeffd640 1 -- 172.21.15.33:0/1931341037 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7faae8084dd0 con 0x7faaf806eb60 2024-04-25T01:08:22.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.598+0000 7faaf5d9b640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faad80678c0 0x7faad8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:22.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.598+0000 7faaf5d9b640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faad80678c0 0x7faad8069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faae00af000 tx=0x7faae0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:22.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.602+0000 7faaeeffd640 1 -- 172.21.15.33:0/1931341037 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faae8055b80 con 0x7faaf806eb60 2024-04-25T01:08:22.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:22.762+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7faaf8072ac0 con 0x7faaf806eb60 2024-04-25T01:08:23.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.422+0000 7faaeeffd640 1 -- 172.21.15.33:0/1931341037 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7faae8086070 con 0x7faaf806eb60 2024-04-25T01:08:23.416 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-04-25T01:08:23.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faad80678c0 msgr2=0x7faad8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:23.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faad80678c0 0x7faad8069d80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7faae00af000 tx=0x7faae0008040 comp rx=0 tx=0).stop 2024-04-25T01:08:23.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 msgr2=0x7faaf81593d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:23.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf81593d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faae800c3c0 tx=0x7faae800c880 comp rx=0 tx=0).stop 2024-04-25T01:08:23.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 shutdown_connections 2024-04-25T01:08:23.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faad80678c0 0x7faad8069d80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faaf8145660 0x7faaf8143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faaf8071170 0x7faaf80aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 --2- 172.21.15.33:0/1931341037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faaf806eb60 0x7faaf81593d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 >> 172.21.15.33:0/1931341037 conn(0x7faaf80b1810 msgr2=0x7faaf8142f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:23.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 shutdown_connections 2024-04-25T01:08:23.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.426+0000 7faafcf55640 1 -- 172.21.15.33:0/1931341037 wait complete. 2024-04-25T01:08:23.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T01:08:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00c13ac640 1 Processor -- start 2024-04-25T01:08:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00c13ac640 1 -- start start 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00c13ac640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00bc15f4c0 0x7f00bc06c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00c13ac640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00bc0b5050 con 0x7f00bc15ae40 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00c13ac640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00bc0cd440 con 0x7f00bc0b4b50 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00c13ac640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f00bc15e920 con 0x7f00bc15f4c0 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00baffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f00bc15ae40 0x7f00bc0cea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33740/0 (socket says 172.21.15.33:33740) 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00baffd640 1 -- 172.21.15.33:0/1296831842 learned_addr learned my addr 172.21.15.33:0/1296831842 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00bb7fe640 1 --2- 172.21.15.33:0/1296831842 >> v2:172.21.15.138:3300/0 conn(0x7f00bc15f4c0 0x7f00bc06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 65900999 0 0) 0x7f00bc0b5050 con 0x7f00bc15ae40 2024-04-25T01:08:23.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a8003530 con 0x7f00bc15ae40 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1742227035 0 0) 0x7f00a8003530 con 0x7f00bc15ae40 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 >> v2:172.21.15.138:3300/0 conn(0x7f00bc15f4c0 msgr2=0x7f00bc06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 --2- 172.21.15.33:0/1296831842 >> v2:172.21.15.138:3300/0 conn(0x7f00bc15f4c0 0x7f00bc06c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 >> v1:172.21.15.138:6789/0 conn(0x7f00bc0b4b50 legacy=0x7f00bc15edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc0cce80 con 0x7f00bc15ae40 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f00ac001680 con 0x7f00bc15ae40 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 >> v1:172.21.15.33:6789/0 conn(0x7f00bc15ae40 legacy=0x7f00bc0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00a80039c0 0x7f00a8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00a8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8005090 0x7f00a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc15e920 con 0x7f00a80043c0 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00a800bff0 con 0x7f00a8005090 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00a800c1f0 con 0x7f00a80039c0 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00ba7fc640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00a8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00bb7fe640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8005090 0x7f00a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00baffd640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00a80039c0 0x7f00a8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00ba7fc640 1 -- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00a80039c0 msgr2=0x7f00a8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:23.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00ba7fc640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00a80039c0 0x7f00a8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00ba7fc640 1 -- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8005090 msgr2=0x7f00a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:23.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00ba7fc640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00a8005090 0x7f00a800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00ba7fc640 1 -- 172.21.15.33:0/1296831842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a8003530 con 0x7f00a80043c0 2024-04-25T01:08:23.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.586+0000 7f00bb7fe640 1 -- 172.21.15.33:0/1296831842 reap_dead start 2024-04-25T01:08:23.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00ba7fc640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00a8004890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00a4001f60 tx=0x7f00a401efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981d3b5c6dd5e5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a40b5070 con 0x7f00a80043c0 2024-04-25T01:08:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00a40a8bf0 con 0x7f00a80043c0 2024-04-25T01:08:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00b9ffb640 1 -- 172.21.15.33:0/1296831842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a4001b60 con 0x7f00a80043c0 2024-04-25T01:08:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 -- 172.21.15.33:0/1296831842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 msgr2=0x7f00a8004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00a8004890 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00a4001f60 tx=0x7f00a401efd0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 -- 172.21.15.33:0/1296831842 shutdown_connections 2024-04-25T01:08:23.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 --2- 172.21.15.33:0/1296831842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00a8004890 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 -- 172.21.15.33:0/1296831842 >> 172.21.15.33:0/1296831842 conn(0x7f00bc0715d0 msgr2=0x7f00bc0719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 -- 172.21.15.33:0/1296831842 shutdown_connections 2024-04-25T01:08:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 -- 172.21.15.33:0/1296831842 wait complete. 2024-04-25T01:08:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 Processor -- start 2024-04-25T01:08:23.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.590+0000 7f00c13ac640 1 -- start start 2024-04-25T01:08:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00bc06b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00bc06b750 0x7f00bc06d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00bc06db70 0x7f00bc06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc06f3d0 con 0x7f00a80043c0 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc06f250 con 0x7f00bc06db70 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc0b52c0 con 0x7f00bc06b750 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00bc06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00bc06b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44618/0 (socket says 172.21.15.33:44618) 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 -- 172.21.15.33:0/2686792399 learned_addr learned my addr 172.21.15.33:0/2686792399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00ba7fc640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00bc06b750 0x7f00bc06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00bb7fe640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00bc06db70 0x7f00bc06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 -- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00bc06b750 msgr2=0x7f00bc06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00bc06b750 0x7f00bc06d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 -- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00bc06db70 msgr2=0x7f00bc06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00bc06db70 0x7f00bc06dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 -- 172.21.15.33:0/2686792399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc0a9430 con 0x7f00a80043c0 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00bb7fe640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00bc06db70 0x7f00bc06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00baffd640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00bc06b210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e850 tx=0x7f00ac00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00b37fe640 1 -- 172.21.15.33:0/2686792399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00c870 con 0x7f00a80043c0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00b37fe640 1 -- 172.21.15.33:0/2686792399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00ac018070 con 0x7f00a80043c0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc14b4c0 con 0x7f00a80043c0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00b37fe640 1 -- 172.21.15.33:0/2686792399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00cc30 con 0x7f00a80043c0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc0a99d0 con 0x7f00a80043c0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc069040 con 0x7f00a80043c0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00b37fe640 1 -- 172.21.15.33:0/2686792399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00ac013420 con 0x7f00a80043c0 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00b37fe640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f009c0678f0 0x7f009c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00b37fe640 1 -- 172.21.15.33:0/2686792399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f00ac0857a0 con 0x7f00a80043c0 2024-04-25T01:08:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.594+0000 7f00ba7fc640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f009c0678f0 0x7f009c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:23.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.598+0000 7f00ba7fc640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f009c0678f0 0x7f009c069db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f00a40b2000 tx=0x7f00a40aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:23.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.602+0000 7f00b37fe640 1 -- 172.21.15.33:0/2686792399 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00ac056340 con 0x7f00a80043c0 2024-04-25T01:08:23.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:23.798+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f00bc156420 con 0x7f00a80043c0 2024-04-25T01:08:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.441+0000 7f00b37fe640 1 -- 172.21.15.33:0/2686792399 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f00ac055c40 con 0x7f00a80043c0 2024-04-25T01:08:24.434 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T01:08:24.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f009c0678f0 msgr2=0x7f009c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:24.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f009c0678f0 0x7f009c069db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f00a40b2000 tx=0x7f00a40aa070 comp rx=0 tx=0).stop 2024-04-25T01:08:24.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 msgr2=0x7f00bc06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:24.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00bc06b210 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f00ac00e850 tx=0x7f00ac00ed10 comp rx=0 tx=0).stop 2024-04-25T01:08:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 shutdown_connections 2024-04-25T01:08:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f009c0678f0 0x7f009c069db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00bc06db70 0x7f00bc06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00bc06b750 0x7f00bc06d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 --2- 172.21.15.33:0/2686792399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00a80043c0 0x7f00bc06b210 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 >> 172.21.15.33:0/2686792399 conn(0x7f00bc0715d0 msgr2=0x7f00bc15a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:24.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 shutdown_connections 2024-04-25T01:08:24.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.445+0000 7f00c13ac640 1 -- 172.21.15.33:0/2686792399 wait complete. 2024-04-25T01:08:24.476 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T01:08:24.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T01:08:24.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.613+0000 7f3697557640 1 Processor -- start 2024-04-25T01:08:24.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.613+0000 7f3697557640 1 -- start start 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3697557640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f369006ca40 0x7f36900727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3697557640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369015efa0 con 0x7f3690072cf0 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3697557640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f369015f280 con 0x7f3690144700 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3697557640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3690072040 con 0x7f369006ca40 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3695af4640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3690072cf0 0x7f36900cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33742/0 (socket says 172.21.15.33:33742) 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3695af4640 1 -- 172.21.15.33:0/584874410 learned_addr learned my addr 172.21.15.33:0/584874410 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3694af2640 1 --2- 172.21.15.33:0/584874410 >> v2:172.21.15.138:3300/0 conn(0x7f369006ca40 0x7f36900727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148840382 0 0) 0x7f369015efa0 con 0x7f3690072cf0 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f367c003530 con 0x7f3690072cf0 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2331440949 0 0) 0x7f367c003530 con 0x7f3690072cf0 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 >> v2:172.21.15.138:3300/0 conn(0x7f369006ca40 msgr2=0x7f36900727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 --2- 172.21.15.33:0/584874410 >> v2:172.21.15.138:3300/0 conn(0x7f369006ca40 0x7f36900727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 >> v1:172.21.15.138:6789/0 conn(0x7f3690144700 legacy=0x7f369015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f369015f560 con 0x7f3690072cf0 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f3688001680 con 0x7f3690072cf0 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 >> v1:172.21.15.33:6789/0 conn(0x7f3690072cf0 legacy=0x7f36900cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f367c0039c0 0x7f367c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c004810 0x7f367c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0051d0 0x7f367c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3690072040 con 0x7f367c0039c0 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f367c00c070 con 0x7f367c004810 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f367c00c270 con 0x7f367c0051d0 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f36952f3640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f367c0039c0 0x7f367c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3694af2640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c004810 0x7f367c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3695af4640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0051d0 0x7f367c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f36952f3640 1 -- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0051d0 msgr2=0x7f367c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f36952f3640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0051d0 0x7f367c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f36952f3640 1 -- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c004810 msgr2=0x7f367c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f36952f3640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c004810 0x7f367c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f36952f3640 1 -- 172.21.15.33:0/584874410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c003530 con 0x7f367c0039c0 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3695af4640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f367c0051d0 0x7f367c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.617+0000 7f3695af4640 1 -- 172.21.15.33:0/584874410 reap_dead start 2024-04-25T01:08:24.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f36952f3640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f367c0039c0 0x7f367c003de0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f368c001e70 tx=0x7f368c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbcd69472643771f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c001a90 con 0x7f367c0039c0 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f368c003b80 con 0x7f367c0039c0 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3687fff640 1 -- 172.21.15.33:0/584874410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368c005080 con 0x7f367c0039c0 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 -- 172.21.15.33:0/584874410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f367c0039c0 msgr2=0x7f367c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f367c0039c0 0x7f367c003de0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f368c001e70 tx=0x7f368c01c730 comp rx=0 tx=0).stop 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 -- 172.21.15.33:0/584874410 shutdown_connections 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 --2- 172.21.15.33:0/584874410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f367c0039c0 0x7f367c003de0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 -- 172.21.15.33:0/584874410 >> 172.21.15.33:0/584874410 conn(0x7f369006b3e0 msgr2=0x7f369006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 -- 172.21.15.33:0/584874410 shutdown_connections 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 -- 172.21.15.33:0/584874410 wait complete. 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 Processor -- start 2024-04-25T01:08:24.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.621+0000 7f3697557640 1 -- start start 2024-04-25T01:08:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0039c0 0x7f369015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36901e73f0 0x7f36901e9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36900b2ba0 0x7f36900b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f36900ccfc0 con 0x7f36900b2ba0 2024-04-25T01:08:24.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f36900cce40 con 0x7f367c0039c0 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f36900cd140 con 0x7f36901e73f0 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36900b2ba0 0x7f36900b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36900b2ba0 0x7f36900b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44646/0 (socket says 172.21.15.33:44646) 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 -- 172.21.15.33:0/1080250143 learned_addr learned my addr 172.21.15.33:0/1080250143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f36952f3640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0039c0 0x7f369015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3694af2640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36901e73f0 0x7f36901e9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 -- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36901e73f0 msgr2=0x7f36901e9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36901e73f0 0x7f36901e9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 -- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0039c0 msgr2=0x7f369015a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0039c0 0x7f369015a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 -- 172.21.15.33:0/1080250143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36901e7870 con 0x7f36900b2ba0 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3694af2640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36901e73f0 0x7f36901e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3695af4640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36900b2ba0 0x7f36900b3020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3690070fe0 tx=0x7f368800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:24.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3685ffb640 1 -- 172.21.15.33:0/1080250143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688007440 con 0x7f36900b2ba0 2024-04-25T01:08:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3685ffb640 1 -- 172.21.15.33:0/1080250143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36880075e0 con 0x7f36900b2ba0 2024-04-25T01:08:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36901e7b20 con 0x7f36900b2ba0 2024-04-25T01:08:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3685ffb640 1 -- 172.21.15.33:0/1080250143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3688004a10 con 0x7f36900b2ba0 2024-04-25T01:08:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36901e80e0 con 0x7f36900b2ba0 2024-04-25T01:08:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3690069040 con 0x7f36900b2ba0 2024-04-25T01:08:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3685ffb640 1 -- 172.21.15.33:0/1080250143 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3688004bb0 con 0x7f36900b2ba0 2024-04-25T01:08:24.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3685ffb640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f36740678f0 0x7f3674069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:24.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f3685ffb640 1 -- 172.21.15.33:0/1080250143 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3688085450 con 0x7f36900b2ba0 2024-04-25T01:08:24.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.625+0000 7f36952f3640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f36740678f0 0x7f3674069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:24.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.629+0000 7f36952f3640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f36740678f0 0x7f3674069db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f368c005000 tx=0x7f368c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.629+0000 7f3685ffb640 1 -- 172.21.15.33:0/1080250143 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3688055f90 con 0x7f36900b2ba0 2024-04-25T01:08:24.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:24.797+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f369013db30 con 0x7f36900b2ba0 2024-04-25T01:08:25.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.445+0000 7f3685ffb640 1 -- 172.21.15.33:0/1080250143 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f3688055890 con 0x7f36900b2ba0 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f36740678f0 msgr2=0x7f3674069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f36740678f0 0x7f3674069db0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f368c005000 tx=0x7f368c009040 comp rx=0 tx=0).stop 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36900b2ba0 msgr2=0x7f36900b3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36900b2ba0 0x7f36900b3020 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3690070fe0 tx=0x7f368800d960 comp rx=0 tx=0).stop 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 shutdown_connections 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f36740678f0 0x7f3674069db0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36900b2ba0 0x7f36900b3020 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f36901e73f0 0x7f36901e9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 --2- 172.21.15.33:0/1080250143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f367c0039c0 0x7f369015a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 >> 172.21.15.33:0/1080250143 conn(0x7f369006b3e0 msgr2=0x7f36900b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 shutdown_connections 2024-04-25T01:08:25.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.449+0000 7f3697557640 1 -- 172.21.15.33:0/1080250143 wait complete. 2024-04-25T01:08:25.467 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T01:08:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce29000640 1 Processor -- start 2024-04-25T01:08:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce29000640 1 -- start start 2024-04-25T01:08:25.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce29000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fce240cea60 0x7fce240ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce29000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce240b29f0 con 0x7fce2415f8e0 2024-04-25T01:08:25.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce29000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce240b2cd0 con 0x7fce24145530 2024-04-25T01:08:25.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce29000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fce240b3600 con 0x7fce240cea60 2024-04-25T01:08:25.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce22d9d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fce2415f8e0 0x7fce2406cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33754/0 (socket says 172.21.15.33:33754) 2024-04-25T01:08:25.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce22d9d640 1 -- 172.21.15.33:0/3092581875 learned_addr learned my addr 172.21.15.33:0/3092581875 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:25.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce19d9b640 1 --2- 172.21.15.33:0/3092581875 >> v2:172.21.15.138:3300/0 conn(0x7fce240cea60 0x7fce240ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836297845 0 0) 0x7fce240b29f0 con 0x7fce2415f8e0 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce10003530 con 0x7fce2415f8e0 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 587442023 0 0) 0x7fce10003530 con 0x7fce2415f8e0 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 >> v2:172.21.15.138:3300/0 conn(0x7fce240cea60 msgr2=0x7fce240ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 --2- 172.21.15.33:0/3092581875 >> v2:172.21.15.138:3300/0 conn(0x7fce240cea60 0x7fce240ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 >> v1:172.21.15.138:6789/0 conn(0x7fce24145530 legacy=0x7fce240b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce240b2fb0 con 0x7fce2415f8e0 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fce1c001a00 con 0x7fce2415f8e0 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 >> v1:172.21.15.33:6789/0 conn(0x7fce2415f8e0 legacy=0x7fce2406cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100039c0 0x7fce10003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce10004810 0x7fce10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100051d0 0x7fce1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce240b3600 con 0x7fce100039c0 2024-04-25T01:08:25.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce1000c0a0 con 0x7fce10004810 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce1000c2a0 con 0x7fce100051d0 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce2259c640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100039c0 0x7fce10003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce19d9b640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce10004810 0x7fce10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce22d9d640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100051d0 0x7fce1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce2259c640 1 -- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100051d0 msgr2=0x7fce1000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce2259c640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100051d0 0x7fce1000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce2259c640 1 -- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce10004810 msgr2=0x7fce10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce2259c640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce10004810 0x7fce10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce2259c640 1 -- 172.21.15.33:0/3092581875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce10003530 con 0x7fce100039c0 2024-04-25T01:08:25.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.613+0000 7fce22d9d640 1 -- 172.21.15.33:0/3092581875 reap_dead start 2024-04-25T01:08:25.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce2259c640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100039c0 0x7fce10003e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fce14001780 tx=0x7fce1401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6257047d7474f03f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce140013a0 con 0x7fce100039c0 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce140052f0 con 0x7fce100039c0 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce21d9b640 1 -- 172.21.15.33:0/3092581875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce140a4e60 con 0x7fce100039c0 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- 172.21.15.33:0/3092581875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100039c0 msgr2=0x7fce10003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100039c0 0x7fce10003e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fce14001780 tx=0x7fce1401c3d0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- 172.21.15.33:0/3092581875 shutdown_connections 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 --2- 172.21.15.33:0/3092581875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100039c0 0x7fce10003e00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- 172.21.15.33:0/3092581875 >> 172.21.15.33:0/3092581875 conn(0x7fce240b1630 msgr2=0x7fce240b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- 172.21.15.33:0/3092581875 shutdown_connections 2024-04-25T01:08:25.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- 172.21.15.33:0/3092581875 wait complete. 2024-04-25T01:08:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 Processor -- start 2024-04-25T01:08:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- start start 2024-04-25T01:08:25.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100039c0 0x7fce240716f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce240aff10 0x7fce241381a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce241386e0 0x7fce240ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce24160100 con 0x7fce241386e0 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce240cf250 con 0x7fce240aff10 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce29000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce240ad8e0 con 0x7fce100039c0 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.617+0000 7fce22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce241386e0 0x7fce240ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce241386e0 0x7fce240ae6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44662/0 (socket says 172.21.15.33:44662) 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 -- 172.21.15.33:0/1208355397 learned_addr learned my addr 172.21.15.33:0/1208355397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce19d9b640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce240aff10 0x7fce241381a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce2259c640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100039c0 0x7fce240716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 -- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100039c0 msgr2=0x7fce240716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100039c0 0x7fce240716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 -- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce240aff10 msgr2=0x7fce241381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce240aff10 0x7fce241381a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 -- 172.21.15.33:0/1208355397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce2415f300 con 0x7fce241386e0 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce19d9b640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce240aff10 0x7fce241381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce2259c640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100039c0 0x7fce240716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce22d9d640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce241386e0 0x7fce240ae6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce1c001590 tx=0x7fce1c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce1b7fe640 1 -- 172.21.15.33:0/1208355397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c019070 con 0x7fce241386e0 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce1b7fe640 1 -- 172.21.15.33:0/1208355397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce1c00bb20 con 0x7fce241386e0 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce240adbc0 con 0x7fce241386e0 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce1b7fe640 1 -- 172.21.15.33:0/1208355397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00dea0 con 0x7fce241386e0 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce240ae180 con 0x7fce241386e0 2024-04-25T01:08:25.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce240ae380 con 0x7fce241386e0 2024-04-25T01:08:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce1b7fe640 1 -- 172.21.15.33:0/1208355397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce1c007030 con 0x7fce241386e0 2024-04-25T01:08:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce1b7fe640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fce040678c0 0x7fce04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce1b7fe640 1 -- 172.21.15.33:0/1208355397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7fce1c085800 con 0x7fce241386e0 2024-04-25T01:08:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce2259c640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fce040678c0 0x7fce04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:25.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.621+0000 7fce2259c640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fce040678c0 0x7fce04069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce140ac000 tx=0x7fce140011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:25.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.625+0000 7fce1b7fe640 1 -- 172.21.15.33:0/1208355397 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fce240ae380 con 0x7fce241386e0 2024-04-25T01:08:25.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:25.789+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fce24069040 con 0x7fce241386e0 2024-04-25T01:08:27.361 INFO:tasks.ceph.mds.c.smithi033.stderr:2024-04-25T01:08:27.369+0000 7f3087f47640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:08:27.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.489+0000 7fce1b7fe640 1 -- 172.21.15.33:0/1208355397 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7fce1c056210 con 0x7fce241386e0 2024-04-25T01:08:27.483 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' created 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fce040678c0 msgr2=0x7fce04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fce040678c0 0x7fce04069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fce140ac000 tx=0x7fce140011a0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce241386e0 msgr2=0x7fce240ae6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce241386e0 0x7fce240ae6a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fce1c001590 tx=0x7fce1c00b4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 shutdown_connections 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fce040678c0 0x7fce04069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce241386e0 0x7fce240ae6a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce240aff10 0x7fce241381a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 --2- 172.21.15.33:0/1208355397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100039c0 0x7fce240716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 >> 172.21.15.33:0/1208355397 conn(0x7fce240b1630 msgr2=0x7fce241594b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 shutdown_connections 2024-04-25T01:08:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.493+0000 7fce29000640 1 -- 172.21.15.33:0/1208355397 wait complete. 2024-04-25T01:08:27.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T01:08:27.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 Processor -- start 2024-04-25T01:08:27.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 -- start start 2024-04-25T01:08:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7c0144860 0x7fc7c00cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc7c0070460 0x7fc7c00ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c006b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c00708a0 con 0x7fc7c0144860 2024-04-25T01:08:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc7c00ceef0 con 0x7fc7c0070460 2024-04-25T01:08:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7c5f59640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c00cf1d0 con 0x7fc7c006b210 2024-04-25T01:08:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7bf7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7c0144860 0x7fc7c00cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7bf7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc7c0144860 0x7fc7c00cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44676/0 (socket says 172.21.15.33:44676) 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7bf7fe640 1 -- 172.21.15.33:0/3176269154 learned_addr learned my addr 172.21.15.33:0/3176269154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.649+0000 7fc7beffd640 1 --2- 172.21.15.33:0/3176269154 >> v2:172.21.15.138:3300/0 conn(0x7fc7c0070460 0x7fc7c00ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7bf7fe640 1 -- 172.21.15.33:0/3176269154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 msgr2=0x7fc7c006b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7bffff640 1 --2- 172.21.15.33:0/3176269154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c006b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7bf7fe640 1 --2- 172.21.15.33:0/3176269154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c006b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7bf7fe640 1 -- 172.21.15.33:0/3176269154 >> v2:172.21.15.138:3300/0 conn(0x7fc7c0070460 msgr2=0x7fc7c00ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7bf7fe640 1 --2- 172.21.15.33:0/3176269154 >> v2:172.21.15.138:3300/0 conn(0x7fc7c0070460 0x7fc7c00ce6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7bf7fe640 1 -- 172.21.15.33:0/3176269154 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c00cf4b0 con 0x7fc7c0144860 2024-04-25T01:08:27.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7beffd640 1 --2- 172.21.15.33:0/3176269154 >> v2:172.21.15.138:3300/0 conn(0x7fc7c0070460 0x7fc7c00ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7bf7fe640 1 --2- 172.21.15.33:0/3176269154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c00cff90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0071ad0 tx=0x7fc7b001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99474e89cbfd1e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7be7fc640 1 -- 172.21.15.33:0/3176269154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00b1070 con 0x7fc7c0144860 2024-04-25T01:08:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7be7fc640 1 -- 172.21.15.33:0/3176269154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b00ac630 con 0x7fc7c0144860 2024-04-25T01:08:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7be7fc640 1 -- 172.21.15.33:0/3176269154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00a8050 con 0x7fc7c0144860 2024-04-25T01:08:27.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 -- 172.21.15.33:0/3176269154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 msgr2=0x7fc7c00cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:27.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/3176269154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c00cff90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0071ad0 tx=0x7fc7b001c520 comp rx=0 tx=0).stop 2024-04-25T01:08:27.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 -- 172.21.15.33:0/3176269154 shutdown_connections 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/3176269154 >> v2:172.21.15.138:3300/0 conn(0x7fc7c0070460 0x7fc7c00ce6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/3176269154 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c006b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/3176269154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c00cff90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 -- 172.21.15.33:0/3176269154 >> 172.21.15.33:0/3176269154 conn(0x7fc7c00d0af0 msgr2=0x7fc7c00d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 -- 172.21.15.33:0/3176269154 shutdown_connections 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.653+0000 7fc7c5f59640 1 -- 172.21.15.33:0/3176269154 wait complete. 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 Processor -- start 2024-04-25T01:08:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 -- start start 2024-04-25T01:08:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c01476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7c0070460 0x7fc7c00b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c00ae1d0 con 0x7fc7c0144860 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c00ae050 con 0x7fc7c0070460 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c00ae350 con 0x7fc7c006b210 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c015eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44686/0 (socket says 172.21.15.33:44686) 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 -- 172.21.15.33:0/2522556858 learned_addr learned my addr 172.21.15.33:0/2522556858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7beffd640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7c0070460 0x7fc7c00b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bf7fe640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c01476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 -- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 msgr2=0x7fc7c01476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c01476f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 -- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7c0070460 msgr2=0x7fc7c00b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7c0070460 0x7fc7c00b3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 -- 172.21.15.33:0/2522556858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c015f050 con 0x7fc7c0144860 2024-04-25T01:08:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7bffff640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c015eb10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800cd70 tx=0x7fc7b800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7b7fff640 1 -- 172.21.15.33:0/2522556858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8003ba0 con 0x7fc7c0144860 2024-04-25T01:08:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7b7fff640 1 -- 172.21.15.33:0/2522556858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b8003d40 con 0x7fc7c0144860 2024-04-25T01:08:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c013feb0 con 0x7fc7c0144860 2024-04-25T01:08:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7b7fff640 1 -- 172.21.15.33:0/2522556858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8004240 con 0x7fc7c0144860 2024-04-25T01:08:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c0140470 con 0x7fc7c0144860 2024-04-25T01:08:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c0069040 con 0x7fc7c0144860 2024-04-25T01:08:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.657+0000 7fc7b7fff640 1 -- 172.21.15.33:0/2522556858 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc7b80043e0 con 0x7fc7c0144860 2024-04-25T01:08:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.661+0000 7fc7b7fff640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc7a00678c0 0x7fc7a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.661+0000 7fc7b7fff640 1 -- 172.21.15.33:0/2522556858 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fc7b8086a30 con 0x7fc7c0144860 2024-04-25T01:08:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.661+0000 7fc7bf7fe640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc7a00678c0 0x7fc7a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:27.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.661+0000 7fc7bf7fe640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc7a00678c0 0x7fc7a0069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00ae000 tx=0x7fc7b00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.665+0000 7fc7b7fff640 1 -- 172.21.15.33:0/2522556858 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc7b8057250 con 0x7fc7c0144860 2024-04-25T01:08:27.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:27.825+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fc7c013b2a0 con 0x7fc7c0144860 2024-04-25T01:08:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.497+0000 7fc7b7fff640 1 -- 172.21.15.33:0/2522556858 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7fc7b8056b50 con 0x7fc7c0144860 2024-04-25T01:08:28.492 INFO:teuthology.orchestra.run.smithi033.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T01:08:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc7a00678c0 msgr2=0x7fc7a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc7a00678c0 0x7fc7a0069d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00ae000 tx=0x7fc7b00a7000 comp rx=0 tx=0).stop 2024-04-25T01:08:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 msgr2=0x7fc7c015eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:28.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c015eb10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc7b800cd70 tx=0x7fc7b800a660 comp rx=0 tx=0).stop 2024-04-25T01:08:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 shutdown_connections 2024-04-25T01:08:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc7a00678c0 0x7fc7a0069d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7c0144860 0x7fc7c015eb10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7c0070460 0x7fc7c00b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 --2- 172.21.15.33:0/2522556858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc7c006b210 0x7fc7c01476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.505+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 >> 172.21.15.33:0/2522556858 conn(0x7fc7c00d0af0 msgr2=0x7fc7c00b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:28.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.509+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 shutdown_connections 2024-04-25T01:08:28.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.509+0000 7fc7c5f59640 1 -- 172.21.15.33:0/2522556858 wait complete. 2024-04-25T01:08:28.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T01:08:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0970d01640 1 Processor -- start 2024-04-25T01:08:28.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0970d01640 1 -- start start 2024-04-25T01:08:28.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0970d01640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f096c0b5580 0x7f096c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0970d01640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096c0b05a0 con 0x7f096c0b50d0 2024-04-25T01:08:28.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0970d01640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f096c0b5ea0 con 0x7f096c15d5d0 2024-04-25T01:08:28.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0970d01640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f096c0af500 con 0x7f096c0b5580 2024-04-25T01:08:28.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0969d9b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f096c0b50d0 0x7f096c15ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59342/0 (socket says 172.21.15.33:59342) 2024-04-25T01:08:28.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f0969d9b640 1 -- 172.21.15.33:0/1863607628 learned_addr learned my addr 172.21.15.33:0/1863607628 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:28.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096a59c640 1 --2- 172.21.15.33:0/1863607628 >> v2:172.21.15.138:3300/0 conn(0x7f096c0b5580 0x7f096c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3360886021 0 0) 0x7f0960001120 con 0x7f096c0b50d0 2024-04-25T01:08:28.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0954003530 con 0x7f096c0b50d0 2024-04-25T01:08:28.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2607761057 0 0) 0x7f0954003530 con 0x7f096c0b50d0 2024-04-25T01:08:28.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 >> v2:172.21.15.138:3300/0 conn(0x7f096c0b5580 msgr2=0x7f096c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 --2- 172.21.15.33:0/1863607628 >> v2:172.21.15.138:3300/0 conn(0x7f096c0b5580 0x7f096c0b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 >> v1:172.21.15.138:6789/0 conn(0x7f096c15d5d0 legacy=0x7f096c15da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c0af7e0 con 0x7f096c0b50d0 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f0960001850 con 0x7f096c0b50d0 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 >> v1:172.21.15.33:6789/0 conn(0x7f096c0b50d0 legacy=0x7f096c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09540039c0 0x7f0954003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0954004810 0x7f0954004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540051d0 0x7f095400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0af500 con 0x7f09540039c0 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f095400c070 con 0x7f09540051d0 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.681+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f095400c270 con 0x7f0954004810 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096a59c640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09540039c0 0x7f0954003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0969d9b640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0954004810 0x7f0954004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096ad9d640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540051d0 0x7f095400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096a59c640 1 -- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0954004810 msgr2=0x7f0954004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096a59c640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0954004810 0x7f0954004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096a59c640 1 -- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540051d0 msgr2=0x7f095400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096a59c640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540051d0 0x7f095400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096a59c640 1 -- 172.21.15.33:0/1863607628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0954003530 con 0x7f09540039c0 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096ad9d640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540051d0 0x7f095400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:28.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096ad9d640 1 -- 172.21.15.33:0/1863607628 reap_dead start 2024-04-25T01:08:28.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096a59c640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09540039c0 0x7f0954003de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f095c009ae0 tx=0x7f095c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1dc6f2dd6d4d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0096b0 con 0x7f09540039c0 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f095c0a9440 con 0x7f09540039c0 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f096959a640 1 -- 172.21.15.33:0/1863607628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095c0a9830 con 0x7f09540039c0 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0970d01640 1 -- 172.21.15.33:0/1863607628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09540039c0 msgr2=0x7f0954003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0970d01640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09540039c0 0x7f0954003de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f095c009ae0 tx=0x7f095c01ca70 comp rx=0 tx=0).stop 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0970d01640 1 -- 172.21.15.33:0/1863607628 shutdown_connections 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0970d01640 1 --2- 172.21.15.33:0/1863607628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09540039c0 0x7f0954003de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0970d01640 1 -- 172.21.15.33:0/1863607628 >> 172.21.15.33:0/1863607628 conn(0x7f096c09a9d0 msgr2=0x7f096c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0970d01640 1 -- 172.21.15.33:0/1863607628 shutdown_connections 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.685+0000 7f0970d01640 1 -- 172.21.15.33:0/1863607628 wait complete. 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 Processor -- start 2024-04-25T01:08:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 -- start start 2024-04-25T01:08:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540039c0 0x7f096c1ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f096c1e8fa0 0x7f096c0d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f096c15a880 0x7f096c13b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f096c15d250 con 0x7f096c15a880 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f096c0af1b0 con 0x7f09540039c0 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f096c15d3d0 con 0x7f096c1e8fa0 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f096c15a880 0x7f096c13b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f096c15a880 0x7f096c13b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34274/0 (socket says 172.21.15.33:34274) 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 -- 172.21.15.33:0/1479296225 learned_addr learned my addr 172.21.15.33:0/1479296225 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0969d9b640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f096c1e8fa0 0x7f096c0d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096a59c640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540039c0 0x7f096c1ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 -- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f096c1e8fa0 msgr2=0x7f096c0d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f096c1e8fa0 0x7f096c0d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 -- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540039c0 msgr2=0x7f096c1ea2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540039c0 0x7f096c1ea2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:28.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 -- 172.21.15.33:0/1479296225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096c1e9a60 con 0x7f096c15a880 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f096ad9d640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f096c15a880 0x7f096c13b5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f096400d540 tx=0x7f096400da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f095affd640 1 -- 172.21.15.33:0/1479296225 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964018070 con 0x7f096c15a880 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f095affd640 1 -- 172.21.15.33:0/1479296225 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0964003db0 con 0x7f096c15a880 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096c1e9780 con 0x7f096c15a880 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f095affd640 1 -- 172.21.15.33:0/1479296225 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0964005330 con 0x7f096c15a880 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f096c13adb0 con 0x7f096c15a880 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c069040 con 0x7f096c15a880 2024-04-25T01:08:28.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.689+0000 7f095affd640 1 -- 172.21.15.33:0/1479296225 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f096401b070 con 0x7f096c15a880 2024-04-25T01:08:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.693+0000 7f095affd640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f094c0678c0 0x7f094c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.693+0000 7f095affd640 1 -- 172.21.15.33:0/1479296225 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f0964085550 con 0x7f096c15a880 2024-04-25T01:08:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.693+0000 7f096a59c640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f094c0678c0 0x7f094c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:28.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.693+0000 7f096a59c640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f094c0678c0 0x7f094c069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f095c0ae000 tx=0x7f095c0b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:28.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.697+0000 7f095affd640 1 -- 172.21.15.33:0/1479296225 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0964055e00 con 0x7f096c15a880 2024-04-25T01:08:28.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:28.901+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f096c0d1930 con 0x7f096c15a880 2024-04-25T01:08:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.509+0000 7f095affd640 1 -- 172.21.15.33:0/1479296225 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f0964055700 con 0x7f096c15a880 2024-04-25T01:08:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:added data pool 4 to fsmap 2024-04-25T01:08:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f094c0678c0 msgr2=0x7f094c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f094c0678c0 0x7f094c069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f095c0ae000 tx=0x7f095c0b3040 comp rx=0 tx=0).stop 2024-04-25T01:08:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f096c15a880 msgr2=0x7f096c13b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f096c15a880 0x7f096c13b5b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f096400d540 tx=0x7f096400da00 comp rx=0 tx=0).stop 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 shutdown_connections 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f094c0678c0 0x7f094c069d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f096c15a880 0x7f096c13b5b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f096c1e8fa0 0x7f096c0d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 --2- 172.21.15.33:0/1479296225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f09540039c0 0x7f096c1ea2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.513+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 >> 172.21.15.33:0/1479296225 conn(0x7f096c09a9d0 msgr2=0x7f096c1572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.517+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 shutdown_connections 2024-04-25T01:08:29.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.517+0000 7f0970d01640 1 -- 172.21.15.33:0/1479296225 wait complete. 2024-04-25T01:08:29.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.657+0000 7efcf7c1b640 1 Processor -- start 2024-04-25T01:08:29.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.657+0000 7efcf7c1b640 1 -- start start 2024-04-25T01:08:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf7c1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf0145470 0x7efcf015c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf7c1b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efcf00cf0e0 0x7efcf00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf7c1b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf015f9a0 con 0x7efcf00cf540 2024-04-25T01:08:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf7c1b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7efcf015f750 con 0x7efcf00cf0e0 2024-04-25T01:08:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf7c1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efcf015fc80 con 0x7efcf0145470 2024-04-25T01:08:29.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf51b6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efcf00cf0e0 0x7efcf00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf61b8640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efcf00cf540 0x7efcf0071b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59352/0 (socket says 172.21.15.33:59352) 2024-04-25T01:08:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf61b8640 1 -- 172.21.15.33:0/102342767 learned_addr learned my addr 172.21.15.33:0/102342767 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf59b7640 1 --2- 172.21.15.33:0/102342767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf0145470 0x7efcf015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf51b6640 1 -- 172.21.15.33:0/102342767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf0145470 msgr2=0x7efcf015c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf51b6640 1 --2- 172.21.15.33:0/102342767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf0145470 0x7efcf015c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf51b6640 1 -- 172.21.15.33:0/102342767 >> v1:172.21.15.33:6789/0 conn(0x7efcf00cf540 legacy=0x7efcf0071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf51b6640 1 -- 172.21.15.33:0/102342767 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf015ff60 con 0x7efcf00cf0e0 2024-04-25T01:08:29.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf49b5640 1 -- 172.21.15.33:0/102342767 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048710228 0 0) 0x7efcf015f9a0 con 0x7efcf00cf540 2024-04-25T01:08:29.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.661+0000 7efcf51b6640 1 --2- 172.21.15.33:0/102342767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf00cf0e0 0x7efcf00b4e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efce00097c0 tx=0x7efce001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bcb074ced84bac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:29.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf49b5640 1 -- 172.21.15.33:0/102342767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce00b1070 con 0x7efcf00cf0e0 2024-04-25T01:08:29.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf49b5640 1 -- 172.21.15.33:0/102342767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce00ac410 con 0x7efcf00cf0e0 2024-04-25T01:08:29.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf49b5640 1 -- 172.21.15.33:0/102342767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce00ac7e0 con 0x7efcf00cf0e0 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- 172.21.15.33:0/102342767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf00cf0e0 msgr2=0x7efcf00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/102342767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf00cf0e0 0x7efcf00b4e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efce00097c0 tx=0x7efce001c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- 172.21.15.33:0/102342767 shutdown_connections 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/102342767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf00cf0e0 0x7efcf00b4e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/102342767 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf0145470 0x7efcf015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- 172.21.15.33:0/102342767 >> 172.21.15.33:0/102342767 conn(0x7efcf00b1810 msgr2=0x7efcf00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- 172.21.15.33:0/102342767 shutdown_connections 2024-04-25T01:08:29.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- 172.21.15.33:0/102342767 wait complete. 2024-04-25T01:08:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 Processor -- start 2024-04-25T01:08:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- start start 2024-04-25T01:08:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf00cf0e0 0x7efcf014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcf00cf540 0x7efcf00b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf0145470 0x7efcf0143160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0168c70 con 0x7efcf0145470 2024-04-25T01:08:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0168af0 con 0x7efcf00cf540 2024-04-25T01:08:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efcf0168df0 con 0x7efcf00cf0e0 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf0145470 0x7efcf0143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf0145470 0x7efcf0143160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34298/0 (socket says 172.21.15.33:34298) 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 -- 172.21.15.33:0/382623271 learned_addr learned my addr 172.21.15.33:0/382623271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf59b7640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf00cf0e0 0x7efcf014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf51b6640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcf00cf540 0x7efcf00b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 -- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf00cf0e0 msgr2=0x7efcf014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf00cf0e0 0x7efcf014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 -- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcf00cf540 msgr2=0x7efcf00b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcf00cf540 0x7efcf00b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 -- 172.21.15.33:0/382623271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf00b0a50 con 0x7efcf0145470 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf59b7640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf00cf0e0 0x7efcf014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf61b8640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf0145470 0x7efcf0143160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efce800c730 tx=0x7efce800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:29.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efce67fc640 1 -- 172.21.15.33:0/382623271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8018070 con 0x7efcf0145470 2024-04-25T01:08:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efce67fc640 1 -- 172.21.15.33:0/382623271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce800ce40 con 0x7efcf0145470 2024-04-25T01:08:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.665+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf00b0d30 con 0x7efcf0145470 2024-04-25T01:08:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efce67fc640 1 -- 172.21.15.33:0/382623271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce800a800 con 0x7efcf0145470 2024-04-25T01:08:29.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf00b35c0 con 0x7efcf0145470 2024-04-25T01:08:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf0069040 con 0x7efcf0145470 2024-04-25T01:08:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efce67fc640 1 -- 172.21.15.33:0/382623271 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efce801b070 con 0x7efcf0145470 2024-04-25T01:08:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efce67fc640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efcd40678c0 0x7efcd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efce67fc640 1 -- 172.21.15.33:0/382623271 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efce80855d0 con 0x7efcf0145470 2024-04-25T01:08:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efcf59b7640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efcd40678c0 0x7efcd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:29.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.669+0000 7efcf59b7640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efcd40678c0 0x7efcd4069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efcec009840 tx=0x7efcec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:29.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.673+0000 7efce67fc640 1 -- 172.21.15.33:0/382623271 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efce8055d30 con 0x7efcf0145470 2024-04-25T01:08:29.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.873+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efcf0071420 con 0x7efcf0145470 2024-04-25T01:08:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efce67fc640 1 -- 172.21.15.33:0/382623271 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7efce8055630 con 0x7efcf0145470 2024-04-25T01:08:29.868 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:29.869 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:28.906955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:08:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efcd40678c0 msgr2=0x7efcd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efcd40678c0 0x7efcd4069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efcec009840 tx=0x7efcec008040 comp rx=0 tx=0).stop 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf0145470 msgr2=0x7efcf0143160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf0145470 0x7efcf0143160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efce800c730 tx=0x7efce800cbf0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 shutdown_connections 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efcd40678c0 0x7efcd4069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcf0145470 0x7efcf0143160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efcf00cf540 0x7efcf00b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 --2- 172.21.15.33:0/382623271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efcf00cf0e0 0x7efcf014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 >> 172.21.15.33:0/382623271 conn(0x7efcf00b1810 msgr2=0x7efcf00b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 shutdown_connections 2024-04-25T01:08:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:29.877+0000 7efcf7c1b640 1 -- 172.21.15.33:0/382623271 wait complete. 2024-04-25T01:08:29.887 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:29.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:08:29.961 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:08:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.017+0000 7f1f60a77640 1 Processor -- start 2024-04-25T01:08:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.017+0000 7f1f60a77640 1 -- start start 2024-04-25T01:08:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f60a77640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1f5c15e810 0x7f1f5c15eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f60a77640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f5c0b4800 con 0x7f1f5c144730 2024-04-25T01:08:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f60a77640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f5c0b4ae0 con 0x7f1f5c06e270 2024-04-25T01:08:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f60a77640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1f5c0b4dc0 con 0x7f1f5c15e810 2024-04-25T01:08:30.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1f5c144730 0x7f1f5c0b5900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59360/0 (socket says 172.21.15.33:59360) 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 -- 172.21.15.33:0/1401485036 learned_addr learned my addr 172.21.15.33:0/1401485036 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f59d9b640 1 --2- 172.21.15.33:0/1401485036 >> v2:172.21.15.138:3300/0 conn(0x7f1f5c15e810 0x7f1f5c15eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228179634 0 0) 0x7f1f5c0b4800 con 0x7f1f5c144730 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f44003530 con 0x7f1f5c144730 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1720987325 0 0) 0x7f1f44003530 con 0x7f1f5c144730 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 >> v2:172.21.15.138:3300/0 conn(0x7f1f5c15e810 msgr2=0x7f1f5c15eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 --2- 172.21.15.33:0/1401485036 >> v2:172.21.15.138:3300/0 conn(0x7f1f5c15e810 0x7f1f5c15eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 >> v1:172.21.15.138:6789/0 conn(0x7f1f5c06e270 legacy=0x7f1f5c06e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c06dd50 con 0x7f1f5c144730 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f1f4c001680 con 0x7f1f5c144730 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 >> v1:172.21.15.33:6789/0 conn(0x7f1f5c144730 legacy=0x7f1f5c0b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f440039c0 0x7f1f44003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44004810 0x7f1f44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440051d0 0x7f1f4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c0b4dc0 con 0x7f1f440039c0 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400c0a0 con 0x7f1f44004810 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4400c2a0 con 0x7f1f440051d0 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f440039c0 0x7f1f44003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f59d9b640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44004810 0x7f1f44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 -- 172.21.15.33:0/1401485036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440051d0 msgr2=0x7f1f4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440051d0 0x7f1f4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 -- 172.21.15.33:0/1401485036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44004810 msgr2=0x7f1f44004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f44004810 0x7f1f44004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 -- 172.21.15.33:0/1401485036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44003530 con 0x7f1f440039c0 2024-04-25T01:08:30.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5ad9d640 1 -- 172.21.15.33:0/1401485036 reap_dead start 2024-04-25T01:08:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f440039c0 0x7f1f44003da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c2c0 tx=0x7f1f4c01cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddc7d4f75dd4e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c007d60 con 0x7f1f440039c0 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f4c0a9070 con 0x7f1f440039c0 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.021+0000 7f1f5959a640 1 -- 172.21.15.33:0/1401485036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c005d40 con 0x7f1f440039c0 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- 172.21.15.33:0/1401485036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f440039c0 msgr2=0x7f1f44003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f440039c0 0x7f1f44003da0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c2c0 tx=0x7f1f4c01cd40 comp rx=0 tx=0).stop 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- 172.21.15.33:0/1401485036 shutdown_connections 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1401485036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f440039c0 0x7f1f44003da0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- 172.21.15.33:0/1401485036 >> 172.21.15.33:0/1401485036 conn(0x7f1f5c0b29c0 msgr2=0x7f1f5c0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- 172.21.15.33:0/1401485036 shutdown_connections 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- 172.21.15.33:0/1401485036 wait complete. 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 Processor -- start 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- start start 2024-04-25T01:08:30.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440039c0 0x7f1f5c1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f5c13f530 0x7f1f5c09cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0b3ab0 0x7f1f5c0b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c06e650 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1f5c15f6a0 con 0x7f1f5c13f530 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1f5c06e7d0 con 0x7f1f440039c0 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0b3ab0 0x7f1f5c0b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0b3ab0 0x7f1f5c0b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34328/0 (socket says 172.21.15.33:34328) 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 -- 172.21.15.33:0/1861093514 learned_addr learned my addr 172.21.15.33:0/1861093514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f59d9b640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f5c13f530 0x7f1f5c09cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440039c0 0x7f1f5c1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 -- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440039c0 msgr2=0x7f1f5c1f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440039c0 0x7f1f5c1f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 -- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f5c13f530 msgr2=0x7f1f5c09cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f5c13f530 0x7f1f5c09cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 -- 172.21.15.33:0/1861093514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f5c09e6d0 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440039c0 0x7f1f5c1f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f5ad9d640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0b3ab0 0x7f1f5c0b3f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0d0be0 tx=0x7f1f5400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f4affd640 1 -- 172.21.15.33:0/1861093514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54017070 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f4affd640 1 -- 172.21.15.33:0/1861093514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f54007070 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5c13f9b0 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f4affd640 1 -- 172.21.15.33:0/1861093514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f54007460 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.025+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5c09d910 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.029+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f5c0b4470 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.029+0000 7f1f4affd640 1 -- 172.21.15.33:0/1861093514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f5401a070 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.029+0000 7f1f4affd640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1f3c0678c0 0x7f1f3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.029+0000 7f1f4affd640 1 -- 172.21.15.33:0/1861093514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1f540852a0 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.029+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1f3c0678c0 0x7f1f3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.029+0000 7f1f5a59c640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1f3c0678c0 0x7f1f3c069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c2c0 tx=0x7f1f4c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.033+0000 7f1f4affd640 1 -- 172.21.15.33:0/1861093514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1f54055aa0 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.189+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f5c073130 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.193+0000 7f1f4affd640 1 -- 172.21.15.33:0/1861093514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f1f540553a0 con 0x7f1f5c0b3ab0 2024-04-25T01:08:30.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:30.187 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1f3c0678c0 msgr2=0x7f1f3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1f3c0678c0 0x7f1f3c069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c2c0 tx=0x7f1f4c0a7070 comp rx=0 tx=0).stop 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0b3ab0 msgr2=0x7f1f5c0b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0b3ab0 0x7f1f5c0b3f30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0d0be0 tx=0x7f1f5400ca50 comp rx=0 tx=0).stop 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 shutdown_connections 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1f3c0678c0 0x7f1f3c069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f5c0b3ab0 0x7f1f5c0b3f30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f5c13f530 0x7f1f5c09cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 --2- 172.21.15.33:0/1861093514 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1f440039c0 0x7f1f5c1f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 >> 172.21.15.33:0/1861093514 conn(0x7f1f5c0b29c0 msgr2=0x7f1f5c143a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 shutdown_connections 2024-04-25T01:08:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.197+0000 7f1f60a77640 1 -- 172.21.15.33:0/1861093514 wait complete. 2024-04-25T01:08:30.205 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:30.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8f461640 1 Processor -- start 2024-04-25T01:08:30.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8f461640 1 -- start start 2024-04-25T01:08:30.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8f461640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e880b4f60 0x7f6e8806fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8f461640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e88070160 con 0x7f6e880b5de0 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8f461640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e88070440 con 0x7f6e88145660 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8f461640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6e88070720 con 0x7f6e880b4f60 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8d9fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6e880b4f60 0x7f6e8806fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8c9fc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6e880b5de0 0x7f6e880af060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59362/0 (socket says 172.21.15.33:59362) 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8c9fc640 1 -- 172.21.15.33:0/1548282100 learned_addr learned my addr 172.21.15.33:0/1548282100 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8d9fe640 1 -- 172.21.15.33:0/1548282100 >> v1:172.21.15.138:6789/0 conn(0x7f6e88145660 legacy=0x7f6e880b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8d9fe640 1 -- 172.21.15.33:0/1548282100 >> v1:172.21.15.33:6789/0 conn(0x7f6e880b5de0 legacy=0x7f6e880af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.377+0000 7f6e8d9fe640 1 -- 172.21.15.33:0/1548282100 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e88070a00 con 0x7f6e880b4f60 2024-04-25T01:08:30.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8d9fe640 1 --2- 172.21.15.33:0/1548282100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e880b4f60 0x7f6e8806fa30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e800097c0 tx=0x7f6e8001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116a5e89f7204568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e7ffff640 1 -- 172.21.15.33:0/1548282100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800b0070 con 0x7f6e880b4f60 2024-04-25T01:08:30.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e7ffff640 1 -- 172.21.15.33:0/1548282100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e800ab600 con 0x7f6e880b4f60 2024-04-25T01:08:30.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e7ffff640 1 -- 172.21.15.33:0/1548282100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800ab9d0 con 0x7f6e880b4f60 2024-04-25T01:08:30.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- 172.21.15.33:0/1548282100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e880b4f60 msgr2=0x7f6e8806fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1548282100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e880b4f60 0x7f6e8806fa30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e800097c0 tx=0x7f6e8001c460 comp rx=0 tx=0).stop 2024-04-25T01:08:30.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- 172.21.15.33:0/1548282100 shutdown_connections 2024-04-25T01:08:30.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1548282100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e880b4f60 0x7f6e8806fa30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- 172.21.15.33:0/1548282100 >> 172.21.15.33:0/1548282100 conn(0x7f6e880734c0 msgr2=0x7f6e8809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:30.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- 172.21.15.33:0/1548282100 shutdown_connections 2024-04-25T01:08:30.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- 172.21.15.33:0/1548282100 wait complete. 2024-04-25T01:08:30.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 Processor -- start 2024-04-25T01:08:30.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- start start 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e880b4f60 0x7f6e88156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e880b5de0 0x7f6e88155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e88145660 0x7f6e8813b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880b0100 con 0x7f6e88145660 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880aff80 con 0x7f6e880b4f60 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8f461640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6e880b0280 con 0x7f6e880b5de0 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8d9fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e88145660 0x7f6e8813b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8d9fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e88145660 0x7f6e8813b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34356/0 (socket says 172.21.15.33:34356) 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.381+0000 7f6e8d9fe640 1 -- 172.21.15.33:0/1133085398 learned_addr learned my addr 172.21.15.33:0/1133085398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d1fd640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e880b4f60 0x7f6e88156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8c9fc640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e880b5de0 0x7f6e88155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d9fe640 1 -- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e880b5de0 msgr2=0x7f6e88155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d9fe640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e880b5de0 0x7f6e88155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d9fe640 1 -- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e880b4f60 msgr2=0x7f6e88156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d9fe640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e880b4f60 0x7f6e88156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d9fe640 1 -- 172.21.15.33:0/1133085398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e88157070 con 0x7f6e88145660 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8c9fc640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e880b5de0 0x7f6e88155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d9fe640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e88145660 0x7f6e8813b460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e800097c0 tx=0x7f6e8001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e7dffb640 1 -- 172.21.15.33:0/1133085398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800b0050 con 0x7f6e88145660 2024-04-25T01:08:30.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e7dffb640 1 -- 172.21.15.33:0/1133085398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e800bc070 con 0x7f6e88145660 2024-04-25T01:08:30.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e7dffb640 1 -- 172.21.15.33:0/1133085398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e800b5830 con 0x7f6e88145660 2024-04-25T01:08:30.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e88157e90 con 0x7f6e88145660 2024-04-25T01:08:30.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e881575f0 con 0x7f6e88145660 2024-04-25T01:08:30.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e88069040 con 0x7f6e88145660 2024-04-25T01:08:30.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e7dffb640 1 -- 172.21.15.33:0/1133085398 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6e800b59d0 con 0x7f6e88145660 2024-04-25T01:08:30.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e7dffb640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6e6c0678c0 0x7f6e6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e7dffb640 1 -- 172.21.15.33:0/1133085398 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6e801214a0 con 0x7f6e88145660 2024-04-25T01:08:30.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d1fd640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6e6c0678c0 0x7f6e6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.385+0000 7f6e8d1fd640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6e6c0678c0 0x7f6e6c069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800b810 tx=0x7f6e78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.389+0000 7f6e7dffb640 1 -- 172.21.15.33:0/1133085398 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6e80016050 con 0x7f6e88145660 2024-04-25T01:08:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.553+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e880d26b0 con 0x7f6e88145660 2024-04-25T01:08:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.553+0000 7f6e7dffb640 1 -- 172.21.15.33:0/1133085398 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f6e800f1c80 con 0x7f6e88145660 2024-04-25T01:08:30.548 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:30.548 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6e6c0678c0 msgr2=0x7f6e6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6e6c0678c0 0x7f6e6c069d80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800b810 tx=0x7f6e78008040 comp rx=0 tx=0).stop 2024-04-25T01:08:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e88145660 msgr2=0x7f6e8813b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e88145660 0x7f6e8813b460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e800097c0 tx=0x7f6e8001aca0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 shutdown_connections 2024-04-25T01:08:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6e6c0678c0 0x7f6e6c069d80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e88145660 0x7f6e8813b460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6e880b5de0 0x7f6e88155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 --2- 172.21.15.33:0/1133085398 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e880b4f60 0x7f6e88156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 >> 172.21.15.33:0/1133085398 conn(0x7f6e880734c0 msgr2=0x7f6e880b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 shutdown_connections 2024-04-25T01:08:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.557+0000 7f6e8f461640 1 -- 172.21.15.33:0/1133085398 wait complete. 2024-04-25T01:08:30.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.705+0000 7f2a63782640 1 Processor -- start 2024-04-25T01:08:30.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a63782640 1 -- start start 2024-04-25T01:08:30.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a63782640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a5c145470 0x7f2a5c15c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a63782640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c15f9a0 con 0x7f2a5c0cf0e0 2024-04-25T01:08:30.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a63782640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a5c15fc80 con 0x7f2a5c0b5530 2024-04-25T01:08:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a63782640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f2a5c15f750 con 0x7f2a5c145470 2024-04-25T01:08:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a6151e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2a5c145470 0x7f2a5c15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a60d1d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2a5c0cf0e0 0x7f2a5c0b4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59366/0 (socket says 172.21.15.33:59366) 2024-04-25T01:08:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a60d1d640 1 -- 172.21.15.33:0/921258849 learned_addr learned my addr 172.21.15.33:0/921258849 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:30.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a6151e640 1 -- 172.21.15.33:0/921258849 >> v1:172.21.15.138:6789/0 conn(0x7f2a5c0b5530 legacy=0x7f2a5c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a6151e640 1 -- 172.21.15.33:0/921258849 >> v1:172.21.15.33:6789/0 conn(0x7f2a5c0cf0e0 legacy=0x7f2a5c0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a6151e640 1 -- 172.21.15.33:0/921258849 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c15ff60 con 0x7f2a5c145470 2024-04-25T01:08:30.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a53fff640 1 -- 172.21.15.33:0/921258849 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894236883 0 0) 0x7f2a5c15f9a0 con 0x7f2a5c0cf0e0 2024-04-25T01:08:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.709+0000 7f2a6151e640 1 --2- 172.21.15.33:0/921258849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c145470 0x7f2a5c15c160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15f8d0 tx=0x7f2a5801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=decd669102df4df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a53fff640 1 -- 172.21.15.33:0/921258849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580ab620 con 0x7f2a5c145470 2024-04-25T01:08:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a53fff640 1 -- 172.21.15.33:0/921258849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a580b0070 con 0x7f2a5c145470 2024-04-25T01:08:30.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a53fff640 1 -- 172.21.15.33:0/921258849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580ab9e0 con 0x7f2a5c145470 2024-04-25T01:08:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- 172.21.15.33:0/921258849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c145470 msgr2=0x7f2a5c15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 --2- 172.21.15.33:0/921258849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c145470 0x7f2a5c15c160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c15f8d0 tx=0x7f2a5801c460 comp rx=0 tx=0).stop 2024-04-25T01:08:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- 172.21.15.33:0/921258849 shutdown_connections 2024-04-25T01:08:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 --2- 172.21.15.33:0/921258849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c145470 0x7f2a5c15c160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- 172.21.15.33:0/921258849 >> 172.21.15.33:0/921258849 conn(0x7f2a5c0b1810 msgr2=0x7f2a5c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:30.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- 172.21.15.33:0/921258849 shutdown_connections 2024-04-25T01:08:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- 172.21.15.33:0/921258849 wait complete. 2024-04-25T01:08:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 Processor -- start 2024-04-25T01:08:30.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- start start 2024-04-25T01:08:30.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c0b5530 0x7f2a5c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a5c0cf0e0 0x7f2a5c13bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a5c145470 0x7f2a5c142390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c169c80 con 0x7f2a5c0b5530 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c169b00 con 0x7f2a5c0cf0e0 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2a5c169e00 con 0x7f2a5c145470 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c0b5530 0x7f2a5c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c0b5530 0x7f2a5c15ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34378/0 (socket says 172.21.15.33:34378) 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 -- 172.21.15.33:0/3462347962 learned_addr learned my addr 172.21.15.33:0/3462347962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a61d1f640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a5c145470 0x7f2a5c142390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a60d1d640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a5c0cf0e0 0x7f2a5c13bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 -- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a5c145470 msgr2=0x7f2a5c142390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a5c145470 0x7f2a5c142390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 -- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a5c0cf0e0 msgr2=0x7f2a5c13bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a5c0cf0e0 0x7f2a5c13bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 -- 172.21.15.33:0/3462347962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c13c4f0 con 0x7f2a5c0b5530 2024-04-25T01:08:30.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a6151e640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c0b5530 0x7f2a5c15ac50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a580ad040 tx=0x7f2a5801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a51ffb640 1 -- 172.21.15.33:0/3462347962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580abd00 con 0x7f2a5c0b5530 2024-04-25T01:08:30.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a51ffb640 1 -- 172.21.15.33:0/3462347962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a580bf410 con 0x7f2a5c0b5530 2024-04-25T01:08:30.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a51ffb640 1 -- 172.21.15.33:0/3462347962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580bf7e0 con 0x7f2a5c0b5530 2024-04-25T01:08:30.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c13c7d0 con 0x7f2a5c0b5530 2024-04-25T01:08:30.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.713+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c13cd90 con 0x7f2a5c0b5530 2024-04-25T01:08:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.717+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a5c069040 con 0x7f2a5c0b5530 2024-04-25T01:08:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.717+0000 7f2a51ffb640 1 -- 172.21.15.33:0/3462347962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2a580bd020 con 0x7f2a5c0b5530 2024-04-25T01:08:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.717+0000 7f2a51ffb640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a400678c0 0x7f2a40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.717+0000 7f2a51ffb640 1 -- 172.21.15.33:0/3462347962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2a581202e0 con 0x7f2a5c0b5530 2024-04-25T01:08:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.717+0000 7f2a60d1d640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a400678c0 0x7f2a40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.717+0000 7f2a60d1d640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a400678c0 0x7f2a40069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c1fa190 tx=0x7f2a4c007b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:30.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.721+0000 7f2a51ffb640 1 -- 172.21.15.33:0/3462347962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2a58016050 con 0x7f2a5c0b5530 2024-04-25T01:08:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.917+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a5c071420 con 0x7f2a5c0b5530 2024-04-25T01:08:30.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.917+0000 7f2a51ffb640 1 -- 172.21.15.33:0/3462347962 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5180 (secure 0 0 0) 0x7f2a580f0b70 con 0x7f2a5c0b5530 2024-04-25T01:08:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:30.911 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:28.906955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:08:30.911 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 7 2024-04-25T01:08:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a400678c0 msgr2=0x7f2a40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a400678c0 0x7f2a40069d80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c1fa190 tx=0x7f2a4c007b80 comp rx=0 tx=0).stop 2024-04-25T01:08:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c0b5530 msgr2=0x7f2a5c15ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c0b5530 0x7f2a5c15ac50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a580ad040 tx=0x7f2a5801aca0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 shutdown_connections 2024-04-25T01:08:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a400678c0 0x7f2a40069d80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2a5c145470 0x7f2a5c142390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a5c0cf0e0 0x7f2a5c13bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 --2- 172.21.15.33:0/3462347962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a5c0b5530 0x7f2a5c15ac50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 >> 172.21.15.33:0/3462347962 conn(0x7f2a5c0b1810 msgr2=0x7f2a5c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 shutdown_connections 2024-04-25T01:08:30.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:30.921+0000 7f2a63782640 1 -- 172.21.15.33:0/3462347962 wait complete. 2024-04-25T01:08:30.930 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.069+0000 7faf70709640 1 Processor -- start 2024-04-25T01:08:31.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.069+0000 7faf70709640 1 -- start start 2024-04-25T01:08:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf70709640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf680b5530 0x7faf68166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf70709640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6815f9a0 con 0x7faf68145470 2024-04-25T01:08:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf70709640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf6815fc80 con 0x7faf6815c160 2024-04-25T01:08:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf70709640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7faf6815f750 con 0x7faf680b5530 2024-04-25T01:08:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6eca6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7faf680b5530 0x7faf68166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6e4a5640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf68145470 0x7faf680cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59376/0 (socket says 172.21.15.33:59376) 2024-04-25T01:08:31.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6e4a5640 1 -- 172.21.15.33:0/1554610673 learned_addr learned my addr 172.21.15.33:0/1554610673 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:31.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6eca6640 1 -- 172.21.15.33:0/1554610673 >> v1:172.21.15.138:6789/0 conn(0x7faf6815c160 legacy=0x7faf680b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6eca6640 1 -- 172.21.15.33:0/1554610673 >> v1:172.21.15.33:6789/0 conn(0x7faf68145470 legacy=0x7faf680cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6eca6640 1 -- 172.21.15.33:0/1554610673 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6815ff60 con 0x7faf680b5530 2024-04-25T01:08:31.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6d4a3640 1 -- 172.21.15.33:0/1554610673 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4057107633 0 0) 0x7faf6815f9a0 con 0x7faf68145470 2024-04-25T01:08:31.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6eca6640 1 --2- 172.21.15.33:0/1554610673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680b5530 0x7faf68166ae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf60009d90 tx=0x7faf6001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e6e8c213dce1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6d4a3640 1 -- 172.21.15.33:0/1554610673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600b2070 con 0x7faf680b5530 2024-04-25T01:08:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6d4a3640 1 -- 172.21.15.33:0/1554610673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf600a59f0 con 0x7faf680b5530 2024-04-25T01:08:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.073+0000 7faf6d4a3640 1 -- 172.21.15.33:0/1554610673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf600a5dc0 con 0x7faf680b5530 2024-04-25T01:08:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- 172.21.15.33:0/1554610673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680b5530 msgr2=0x7faf68166ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 --2- 172.21.15.33:0/1554610673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680b5530 0x7faf68166ae0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faf60009d90 tx=0x7faf6001c930 comp rx=0 tx=0).stop 2024-04-25T01:08:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- 172.21.15.33:0/1554610673 shutdown_connections 2024-04-25T01:08:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 --2- 172.21.15.33:0/1554610673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf680b5530 0x7faf68166ae0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- 172.21.15.33:0/1554610673 >> 172.21.15.33:0/1554610673 conn(0x7faf680b1810 msgr2=0x7faf680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- 172.21.15.33:0/1554610673 shutdown_connections 2024-04-25T01:08:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- 172.21.15.33:0/1554610673 wait complete. 2024-04-25T01:08:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 Processor -- start 2024-04-25T01:08:31.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- start start 2024-04-25T01:08:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf680b5530 0x7faf6813fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf68145470 0x7faf680b2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6815c160 0x7faf6815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf68169840 con 0x7faf68145470 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf6806e850 con 0x7faf6815c160 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faf681699c0 con 0x7faf680b5530 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf68145470 0x7faf680b2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf68145470 0x7faf680b2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34394/0 (socket says 172.21.15.33:34394) 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 -- 172.21.15.33:0/2811227353 learned_addr learned my addr 172.21.15.33:0/2811227353 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6eca6640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6815c160 0x7faf6815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 -- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf680b5530 msgr2=0x7faf6813fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf680b5530 0x7faf6813fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 -- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6815c160 msgr2=0x7faf6815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6815c160 0x7faf6815ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 -- 172.21.15.33:0/2811227353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf681403a0 con 0x7faf68145470 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf6dca4640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf68145470 0x7faf680b2d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c4c0 tx=0x7faf58001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf5effd640 1 -- 172.21.15.33:0/2811227353 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf58017070 con 0x7faf68145470 2024-04-25T01:08:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf5effd640 1 -- 172.21.15.33:0/2811227353 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf58003600 con 0x7faf68145470 2024-04-25T01:08:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf680b48a0 con 0x7faf68145470 2024-04-25T01:08:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf5effd640 1 -- 172.21.15.33:0/2811227353 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf580039f0 con 0x7faf68145470 2024-04-25T01:08:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.077+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf680b4e60 con 0x7faf68145470 2024-04-25T01:08:31.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.081+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf68069040 con 0x7faf68145470 2024-04-25T01:08:31.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.081+0000 7faf5effd640 1 -- 172.21.15.33:0/2811227353 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faf5801a070 con 0x7faf68145470 2024-04-25T01:08:31.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.081+0000 7faf5effd640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf4c0678c0 0x7faf4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.081+0000 7faf5effd640 1 -- 172.21.15.33:0/2811227353 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faf58085350 con 0x7faf68145470 2024-04-25T01:08:31.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.081+0000 7faf6e4a5640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf4c0678c0 0x7faf4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.081+0000 7faf6e4a5640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf4c0678c0 0x7faf4c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf640011f0 tx=0x7faf64007930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.085+0000 7faf5effd640 1 -- 172.21.15.33:0/2811227353 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faf58055b50 con 0x7faf68145470 2024-04-25T01:08:31.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.253+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faf6806e5d0 con 0x7faf68145470 2024-04-25T01:08:31.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.253+0000 7faf5effd640 1 -- 172.21.15.33:0/2811227353 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7faf58055450 con 0x7faf68145470 2024-04-25T01:08:31.247 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:31.247 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:31.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf4c0678c0 msgr2=0x7faf4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf4c0678c0 0x7faf4c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf640011f0 tx=0x7faf64007930 comp rx=0 tx=0).stop 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf68145470 msgr2=0x7faf680b2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf68145470 0x7faf680b2d10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7faf5800c4c0 tx=0x7faf58001e80 comp rx=0 tx=0).stop 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 shutdown_connections 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf4c0678c0 0x7faf4c069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf6815c160 0x7faf6815ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf68145470 0x7faf680b2d10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 --2- 172.21.15.33:0/2811227353 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf680b5530 0x7faf6813fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 >> 172.21.15.33:0/2811227353 conn(0x7faf680b1810 msgr2=0x7faf680b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 shutdown_connections 2024-04-25T01:08:31.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.257+0000 7faf70709640 1 -- 172.21.15.33:0/2811227353 wait complete. 2024-04-25T01:08:31.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:08:23.800275+0000', 'modified': '2024-04-25T01:08:28.906955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4248}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4248': {'gid': 4248, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/486086516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 486086516}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 486086516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4248, 'qdb_cluster': [4248]} 2024-04-25T01:08:31.669 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:08:31.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T01:08:31.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2cab01d640 1 Processor -- start 2024-04-25T01:08:31.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2cab01d640 1 -- start start 2024-04-25T01:08:31.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2cab01d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2ca415e470 0x7f2ca40ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2cab01d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca40d10b0 con 0x7f2ca40d48d0 2024-04-25T01:08:31.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2cab01d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ca40d1390 con 0x7f2ca40d03e0 2024-04-25T01:08:31.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2cab01d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2ca40d1de0 con 0x7f2ca415e470 2024-04-25T01:08:31.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2ca95ba640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2ca40d48d0 0x7f2ca40cef00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59384/0 (socket says 172.21.15.33:59384) 2024-04-25T01:08:31.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.809+0000 7f2ca95ba640 1 -- 172.21.15.33:0/1421521830 learned_addr learned my addr 172.21.15.33:0/1421521830 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/1421521830 >> v2:172.21.15.138:3300/0 conn(0x7f2ca415e470 0x7f2ca40ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240626529 0 0) 0x7f2ca40d10b0 con 0x7f2ca40d48d0 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c8c003530 con 0x7f2ca40d48d0 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1116715366 0 0) 0x7f2c8c003530 con 0x7f2ca40d48d0 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 >> v2:172.21.15.138:3300/0 conn(0x7f2ca415e470 msgr2=0x7f2ca40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 --2- 172.21.15.33:0/1421521830 >> v2:172.21.15.138:3300/0 conn(0x7f2ca415e470 0x7f2ca40ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 >> v1:172.21.15.138:6789/0 conn(0x7f2ca40d03e0 legacy=0x7f2ca40d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca40d1670 con 0x7f2ca40d48d0 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f2c9c001a00 con 0x7f2ca40d48d0 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 >> v1:172.21.15.33:6789/0 conn(0x7f2ca40d48d0 legacy=0x7f2ca40cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0039c0 0x7f2c8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c004810 0x7f2c8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8c0051d0 0x7f2c8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca40d1de0 con 0x7f2c8c0039c0 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c00c170 con 0x7f2c8c004810 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c00c370 con 0x7f2c8c0051d0 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0039c0 0x7f2c8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c004810 0x7f2c8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8c0051d0 0x7f2c8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca8db9640 1 -- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8c0051d0 msgr2=0x7f2c8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8c0051d0 0x7f2c8c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca8db9640 1 -- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c004810 msgr2=0x7f2c8c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c004810 0x7f2c8c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca8db9640 1 -- 172.21.15.33:0/1421521830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c003530 con 0x7f2c8c0039c0 2024-04-25T01:08:31.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2c8c0051d0 0x7f2c8c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:31.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca95ba640 1 -- 172.21.15.33:0/1421521830 reap_dead start 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0039c0 0x7f2c8c003de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c94001780 tx=0x7f2c9401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b392a5fe7357fc74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c940013a0 con 0x7f2c8c0039c0 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c940050d0 con 0x7f2c8c0039c0 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.813+0000 7f2ca37fe640 1 -- 172.21.15.33:0/1421521830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94003600 con 0x7f2c8c0039c0 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- 172.21.15.33:0/1421521830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0039c0 msgr2=0x7f2c8c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0039c0 0x7f2c8c003de0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c94001780 tx=0x7f2c9401c300 comp rx=0 tx=0).stop 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- 172.21.15.33:0/1421521830 shutdown_connections 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 --2- 172.21.15.33:0/1421521830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c8c0039c0 0x7f2c8c003de0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- 172.21.15.33:0/1421521830 >> 172.21.15.33:0/1421521830 conn(0x7f2ca40d3db0 msgr2=0x7f2ca40d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- 172.21.15.33:0/1421521830 shutdown_connections 2024-04-25T01:08:31.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- 172.21.15.33:0/1421521830 wait complete. 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 Processor -- start 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- start start 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c0039c0 0x7f2ca415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41ebc00 0x7f2ca40cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca41ec080 0x7f2ca40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca40d2df0 con 0x7f2ca41ebc00 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca40d2c70 con 0x7f2c8c0039c0 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca40d2f70 con 0x7f2ca41ec080 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41ebc00 0x7f2ca40cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41ebc00 0x7f2ca40cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34412/0 (socket says 172.21.15.33:34412) 2024-04-25T01:08:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 -- 172.21.15.33:0/2212645654 learned_addr learned my addr 172.21.15.33:0/2212645654 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c0039c0 0x7f2ca415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca41ec080 0x7f2ca40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 -- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca41ec080 msgr2=0x7f2ca40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca41ec080 0x7f2ca40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 -- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c0039c0 msgr2=0x7f2ca415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c0039c0 0x7f2ca415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 -- 172.21.15.33:0/2212645654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ca41e9ec0 con 0x7f2ca41ebc00 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca41ec080 0x7f2ca40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41ebc00 0x7f2ca40cd760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c98011660 tx=0x7f2c98011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca17fa640 1 -- 172.21.15.33:0/2212645654 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800f9c0 con 0x7f2ca41ebc00 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca17fa640 1 -- 172.21.15.33:0/2212645654 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9800fb60 con 0x7f2ca41ebc00 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ca41ea1a0 con 0x7f2ca41ebc00 2024-04-25T01:08:31.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2ca17fa640 1 -- 172.21.15.33:0/2212645654 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98015460 con 0x7f2ca41ebc00 2024-04-25T01:08:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.817+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ca41ea760 con 0x7f2ca41ebc00 2024-04-25T01:08:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.821+0000 7f2ca17fa640 1 -- 172.21.15.33:0/2212645654 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2c98015600 con 0x7f2ca41ebc00 2024-04-25T01:08:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.821+0000 7f2ca17fa640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2c900678c0 0x7f2c90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.821+0000 7f2ca17fa640 1 -- 172.21.15.33:0/2212645654 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c98086980 con 0x7f2ca41ebc00 2024-04-25T01:08:31.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.821+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2c900678c0 0x7f2c90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.821+0000 7f2cae196000 1 Processor -- start 2024-04-25T01:08:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.821+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2c900678c0 0x7f2c90069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c94001750 tx=0x7f2c940011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.821+0000 7f2cae196000 1 -- start start 2024-04-25T01:08:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c5491402d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5490f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c5490f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c548f445e0 con 0x55c549140810 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c548ecbc50 con 0x55c5490f6bc0 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c548f405a0 con 0x55c54913fef0 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5490f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5490f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34416/0 (socket says 172.21.15.33:34416) 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 -- 172.21.15.33:0/589764299 learned_addr learned my addr 172.21.15.33:0/589764299 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c5490f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c5491402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 -- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 msgr2=0x55c5491402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c5491402d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 -- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 msgr2=0x55c5490f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c5490f6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 -- 172.21.15.33:0/589764299 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c548ec9d10 con 0x55c549140810 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5490f6680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2c9805f130 tx=0x7f2c98014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd47545536c8f08f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2c8a7fc640 1 -- 172.21.15.33:0/589764299 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9805f660 con 0x55c549140810 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2c8a7fc640 1 -- 172.21.15.33:0/589764299 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c98054d80 con 0x55c549140810 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2c8a7fc640 1 -- 172.21.15.33:0/589764299 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9805f1d0 con 0x55c549140810 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- 172.21.15.33:0/589764299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 msgr2=0x55c5490f6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5490f6680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2c9805f130 tx=0x7f2c98014000 comp rx=0 tx=0).stop 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- 172.21.15.33:0/589764299 shutdown_connections 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c5490f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5490f6680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- 172.21.15.33:0/589764299 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c5491402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- 172.21.15.33:0/589764299 >> 172.21.15.33:0/589764299 conn(0x55c5490f2b70 msgr2=0x55c5490f4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- 172.21.15.33:0/589764299 shutdown_connections 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- 172.21.15.33:0/589764299 wait complete. 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- start start 2024-04-25T01:08:31.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 Processor -- start 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c549124f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c549125460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5491259a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c548efe520 con 0x55c549140810 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c5490f29f0 con 0x55c5490f6bc0 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c548edcdd0 con 0x55c54913fef0 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2cae196000 10 client.0 _ll_register_callbacks cb 0x55c5490e15c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5491259a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5491259a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34418/0 (socket says 172.21.15.33:34418) 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 -- 172.21.15.33:0/2656480403 learned_addr learned my addr 172.21.15.33:0/2656480403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c549125460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c549124f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 -- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 msgr2=0x55c549125460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c549125460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 -- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 msgr2=0x55c549124f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c549124f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 -- 172.21.15.33:0/2656480403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c548edc7b0 con 0x55c549140810 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5491259a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ca408c270 tx=0x7f2c9c00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540e8ef83f5fda27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.825+0000 7f2c837fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 1 -- 172.21.15.33:0/2656480403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c004d80 con 0x55c549140810 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 1 -- 172.21.15.33:0/2656480403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c9c015a60 con 0x55c549140810 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c98054d80 con 0x55c549140810 2024-04-25T01:08:31.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 1 -- 172.21.15.33:0/2656480403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2c9c015c60 con 0x55c549140810 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 10 client.4419 fetch_fsmap learned FSMap version 7 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c548ecbea0 con 0x55c549140810 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 1 -- 172.21.15.33:0/2656480403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c98054d80 con 0x55c549140810 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 1 -- 172.21.15.33:0/2656480403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f2c9c00d6e0 con 0x55c549140810 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 10 client.4419 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 10 client.4419 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 20 client.4419 populate_metadata read hostname 'smithi033' 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c54914ff90 0x55c549152380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 4 client.4419 mds_command: new command op to 4248 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 --> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c548edfab0 con 0x55c54914ff90 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c54914ff90 0x55c549152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c54914ff90 0x55c549152380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=225190c728aa5506 server_cookie=86fd187108ad39ac in_seq=0 out_seq=0 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 10 client.4419 ms_handle_connect on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:31.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 1 -- 172.21.15.33:0/2656480403 <== mds.0 v2:172.21.15.33:6834/486086516 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c548edfab0 con 0x55c54914ff90 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 10 client.4419 handle_command_reply: tid=0 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 client.4419 shutdown 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 msgr2=0x55c5491259a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5491259a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ca408c270 tx=0x7f2c9c00fcc0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 shutdown_connections 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c54914ff90 0x55c549152380 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c549140810 0x55c5491259a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c549125460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 --2- 172.21.15.33:0/2656480403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c5490f6bc0 0x55c549124f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2c837fe640 0 client.4419 ms_handle_reset on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 >> 172.21.15.33:0/2656480403 conn(0x55c5490f2b70 msgr2=0x55c5490f57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 shutdown_connections 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 1 -- 172.21.15.33:0/2656480403 wait complete. 2024-04-25T01:08:31.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.829+0000 7f2cae196000 20 client.4419 trim_cache size 0 max 16384 2024-04-25T01:08:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 Processor -- start 2024-04-25T01:08:31.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- start start 2024-04-25T01:08:31.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5490f6bc0 0x55c5490f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c54913fef0 0x55c5490f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 0x55c5490f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4204660 con 0x55c54913fef0 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2ca4203a40 con 0x55c549140810 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2ca4204010 con 0x55c5490f6bc0 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c54913fef0 0x55c5490f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c54913fef0 0x55c5490f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34434/0 (socket says 172.21.15.33:34434) 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 -- 172.21.15.33:0/1471963369 learned_addr learned my addr 172.21.15.33:0/1471963369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 -- 172.21.15.33:0/1471963369 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5490f6bc0 msgr2=0x55c5490f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5490f6bc0 0x55c5490f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca95ba640 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 0x55c5490f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 -- 172.21.15.33:0/1471963369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 msgr2=0x55c5490f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 0x55c5490f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 -- 172.21.15.33:0/1471963369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c548ecce80 con 0x55c54913fef0 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c54913fef0 0x55c5490f54a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c98052ee0 tx=0x7f2c98059040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb18da366cdc5812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2c89ffb640 1 -- 172.21.15.33:0/1471963369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98083030 con 0x55c54913fef0 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2c89ffb640 1 -- 172.21.15.33:0/1471963369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9805ae30 con 0x55c54913fef0 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2c89ffb640 1 -- 172.21.15.33:0/1471963369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c98054d80 con 0x55c54913fef0 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- 172.21.15.33:0/1471963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c54913fef0 msgr2=0x55c5490f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c54913fef0 0x55c5490f54a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2c98052ee0 tx=0x7f2c98059040 comp rx=0 tx=0).stop 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- 172.21.15.33:0/1471963369 shutdown_connections 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 0x55c5490f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c54913fef0 0x55c5490f54a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 --2- 172.21.15.33:0/1471963369 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c5490f6bc0 0x55c5490f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- 172.21.15.33:0/1471963369 >> 172.21.15.33:0/1471963369 conn(0x55c5490f2b70 msgr2=0x55c54914d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- 172.21.15.33:0/1471963369 shutdown_connections 2024-04-25T01:08:31.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- 172.21.15.33:0/1471963369 wait complete. 2024-04-25T01:08:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 -- start start 2024-04-25T01:08:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.849+0000 7f2cae196000 1 Processor -- start 2024-04-25T01:08:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c5490f6bc0 0x55c5491af410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c549152690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 0x55c549152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c548ef8c40 con 0x55c5490f6bc0 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c548ff6e80 con 0x55c549140810 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55c548f0d810 con 0x55c54913fef0 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 10 client.0 _ll_register_callbacks cb 0x7f2c9c001ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c5490f6bc0 0x55c5491af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c5490f6bc0 0x55c5491af410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34436/0 (socket says 172.21.15.33:34436) 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 -- 172.21.15.33:0/389383758 learned_addr learned my addr 172.21.15.33:0/389383758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 -- 172.21.15.33:0/389383758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 msgr2=0x55c549152690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c549152690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 -- 172.21.15.33:0/389383758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 msgr2=0x55c549152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 0x55c549152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 -- 172.21.15.33:0/389383758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c548f11760 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca8db9640 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c5490f6bc0 0x55c5491af410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c9401c7e0 tx=0x7f2c8c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91371a62ac706878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 1 -- 172.21.15.33:0/389383758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c94007040 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 1 -- 172.21.15.33:0/389383758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2c940ada30 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c9805ae30 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 1 -- 172.21.15.33:0/389383758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f2c940adc30 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 10 client.4425 fetch_fsmap learned FSMap version 7 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55c548f9d9e0 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 1 -- 172.21.15.33:0/389383758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c9805ae30 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 1 -- 172.21.15.33:0/389383758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f2c940071e0 con 0x55c5490f6bc0 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 10 client.4425 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 10 client.4425 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:08:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 20 client.4425 populate_metadata read hostname 'smithi033' 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c549153110 0x55c5491534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 4 client.4425 mds_command: new command op to 4248 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 --> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] -- command(tid 0: {"prefix": "status"}) -- 0x55c548f51b20 con 0x55c549153110 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c549153110 0x55c5491534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2ca3fff640 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c549153110 0x55c5491534f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44af89e4ae4f169a server_cookie=e139160ad0cb77b1 in_seq=0 out_seq=0 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 10 client.4425 ms_handle_connect on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 1 -- 172.21.15.33:0/389383758 <== mds.0 v2:172.21.15.33:6834/486086516 1 ==== command_reply(tid 0: 0 ) ==== 8+0+317 (crc 0 0 0) 0x55c548f51b20 con 0x55c549153110 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 10 client.4425 handle_command_reply: tid=0 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 client.4425 shutdown 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c5490f6bc0 msgr2=0x55c5491af410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c5490f6bc0 0x55c5491af410 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2c9401c7e0 tx=0x7f2c8c005840 comp rx=0 tx=0).stop 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 shutdown_connections 2024-04-25T01:08:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55c549153110 0x55c5491534f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c549140810 0x55c549152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55c54913fef0 0x55c549152690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2c837fe640 0 client.4425 ms_handle_reset on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 --2- 172.21.15.33:0/389383758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c5490f6bc0 0x55c5491af410 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 >> 172.21.15.33:0/389383758 conn(0x55c5490f2b70 msgr2=0x55c54913e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 shutdown_connections 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 1 -- 172.21.15.33:0/389383758 wait complete. 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cae196000 20 client.4425 trim_cache size 0 max 16384 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "10df718f-e6ff-4eb1-9854-6e8a751898ff", 2024-04-25T01:08:31.855 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4248, 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 7.411340332, 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 7, 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 25, 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 14.493990447 2024-04-25T01:08:31.856 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2c900678c0 msgr2=0x7f2c90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2c900678c0 0x7f2c90069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c94001750 tx=0x7f2c940011a0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41ebc00 msgr2=0x7f2ca40cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41ebc00 0x7f2ca40cd760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2c98011660 tx=0x7f2c98011b20 comp rx=0 tx=0).stop 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 shutdown_connections 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2c900678c0 0x7f2c90069d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2ca41ec080 0x7f2ca40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ca41ebc00 0x7f2ca40cd760 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 --2- 172.21.15.33:0/2212645654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2c8c0039c0 0x7f2ca415d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 >> 172.21.15.33:0/2212645654 conn(0x7f2ca40d3db0 msgr2=0x7f2ca4160db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 shutdown_connections 2024-04-25T01:08:31.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.853+0000 7f2cab01d640 1 -- 172.21.15.33:0/2212645654 wait complete. 2024-04-25T01:08:31.864 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:08:31.864 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:31.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:08:31.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.985+0000 7f5ea2858640 1 Processor -- start 2024-04-25T01:08:31.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.985+0000 7f5ea2858640 1 -- start start 2024-04-25T01:08:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea2858640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 0x7f5e9c0725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea2858640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9c15e5d0 con 0x7f5e9c0cfc60 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea2858640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e9c15e8b0 con 0x7f5e9c0b0040 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea2858640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c15eb90 con 0x7f5e9c06b920 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea0df5640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5e9c0cfc60 0x7f5e9c161bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59386/0 (socket says 172.21.15.33:59386) 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea0df5640 1 -- 172.21.15.33:0/3105981456 learned_addr learned my addr 172.21.15.33:0/3105981456 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9b7fe640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 0x7f5e9c0725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660324754 0 0) 0x7f5e9c15e5d0 con 0x7f5e9c0cfc60 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e84003530 con 0x7f5e9c0cfc60 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2147579804 0 0) 0x7f5e84003530 con 0x7f5e9c0cfc60 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 msgr2=0x7f5e9c0725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 0x7f5e9c0725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 >> v1:172.21.15.138:6789/0 conn(0x7f5e9c0b0040 legacy=0x7f5e9c06b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c15ee70 con 0x7f5e9c0cfc60 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f5e94001a00 con 0x7f5e9c0cfc60 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 >> v1:172.21.15.33:6789/0 conn(0x7f5e9c0cfc60 legacy=0x7f5e9c161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 0x7f5e840039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e84003f70 0x7f5e84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e84004d50 0x7f5e840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c15eb90 con 0x7f5e84003f70 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8400bb60 con 0x7f5e84004d50 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5e8400bd60 con 0x7f5e9c06b920 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9b7fe640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e84003f70 0x7f5e84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9bfff640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 0x7f5e840039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea0df5640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e84004d50 0x7f5e840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9b7fe640 1 -- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 msgr2=0x7f5e840039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9b7fe640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e9c06b920 0x7f5e840039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9b7fe640 1 -- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e84004d50 msgr2=0x7f5e840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9b7fe640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e84004d50 0x7f5e840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5e9b7fe640 1 -- 172.21.15.33:0/3105981456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e84003530 con 0x7f5e84003f70 2024-04-25T01:08:31.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.989+0000 7f5ea0df5640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e84004d50 0x7f5e840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:31.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5e9b7fe640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e84003f70 0x7f5e84004810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009ab0 tx=0x7f5e9001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b527e6b1c8bf871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e900b2070 con 0x7f5e84003f70 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e900a5530 con 0x7f5e84003f70 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5e9affd640 1 -- 172.21.15.33:0/3105981456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e900096b0 con 0x7f5e84003f70 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- 172.21.15.33:0/3105981456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e84003f70 msgr2=0x7f5e84004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 --2- 172.21.15.33:0/3105981456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e84003f70 0x7f5e84004810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009ab0 tx=0x7f5e9001ca50 comp rx=0 tx=0).stop 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 -- 172.21.15.33:0/3105981456 reap_dead start 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- 172.21.15.33:0/3105981456 shutdown_connections 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- 172.21.15.33:0/3105981456 >> 172.21.15.33:0/3105981456 conn(0x7f5e9c0b2820 msgr2=0x7f5e9c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- 172.21.15.33:0/3105981456 shutdown_connections 2024-04-25T01:08:31.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- 172.21.15.33:0/3105981456 wait complete. 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 Processor -- start 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- start start 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e84003f70 0x7f5e9c1682b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e9c15baf0 0x7f5e9c06f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e9c146560 0x7f5e9c146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c169d80 con 0x7f5e9c146560 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9c169c00 con 0x7f5e9c15baf0 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea2858640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9c169f00 con 0x7f5e84003f70 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e9c146560 0x7f5e9c146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e9c146560 0x7f5e9c146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34454/0 (socket says 172.21.15.33:34454) 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 -- 172.21.15.33:0/2655632391 learned_addr learned my addr 172.21.15.33:0/2655632391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 -- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e84003f70 msgr2=0x7f5e9c1682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e84003f70 0x7f5e9c1682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 -- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e9c15baf0 msgr2=0x7f5e9c06f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e9c15baf0 0x7f5e9c06f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.993+0000 7f5ea0df5640 1 -- 172.21.15.33:0/2655632391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e9c0d11c0 con 0x7f5e9c146560 2024-04-25T01:08:31.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5ea0df5640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e9c146560 0x7f5e9c146a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e94000f90 tx=0x7f5e9400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e98ff9640 1 -- 172.21.15.33:0/2655632391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400bb20 con 0x7f5e9c146560 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e9c0d14a0 con 0x7f5e9c146560 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e98ff9640 1 -- 172.21.15.33:0/2655632391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9400bce0 con 0x7f5e9c146560 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e98ff9640 1 -- 172.21.15.33:0/2655632391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94007350 con 0x7f5e9c146560 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e9c167d90 con 0x7f5e9c146560 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e9c069040 con 0x7f5e9c146560 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e98ff9640 1 -- 172.21.15.33:0/2655632391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5e9401b020 con 0x7f5e9c146560 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e98ff9640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5e880678f0 0x7f5e88069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:31.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e98ff9640 1 -- 172.21.15.33:0/2655632391 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e940862d0 con 0x7f5e9c146560 2024-04-25T01:08:31.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e9bfff640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5e880678f0 0x7f5e88069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:31.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:31.997+0000 7f5e9bfff640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5e880678f0 0x7f5e88069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c007ab0 tx=0x7f5e8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.001+0000 7f5e98ff9640 1 -- 172.21.15.33:0/2655632391 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5e94056b10 con 0x7f5e9c146560 2024-04-25T01:08:32.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.201+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5e9c06b340 con 0x7f5e9c146560 2024-04-25T01:08:32.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.201+0000 7f5e98ff9640 1 -- 172.21.15.33:0/2655632391 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f5e94056410 con 0x7f5e9c146560 2024-04-25T01:08:32.195 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:08:32.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5e880678f0 msgr2=0x7f5e88069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:32.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5e880678f0 0x7f5e88069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c007ab0 tx=0x7f5e8c009040 comp rx=0 tx=0).stop 2024-04-25T01:08:32.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e9c146560 msgr2=0x7f5e9c146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:32.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e9c146560 0x7f5e9c146a10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5e94000f90 tx=0x7f5e9400b4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 shutdown_connections 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5e880678f0 0x7f5e88069db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e9c146560 0x7f5e9c146a10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e9c15baf0 0x7f5e9c06f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 --2- 172.21.15.33:0/2655632391 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5e84003f70 0x7f5e9c1682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 >> 172.21.15.33:0/2655632391 conn(0x7f5e9c0b2820 msgr2=0x7f5e9c0ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 shutdown_connections 2024-04-25T01:08:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32.205+0000 7f5ea2858640 1 -- 172.21.15.33:0/2655632391 wait complete. 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:08:32.214 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T01:08:32.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:32.215 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:32.221 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42490sec preferred_lft 42490sec 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:26:2b:e6:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:32.222 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T01:08:32.223 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:32.223 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:08:32.223 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:32.223 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:32.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:08:32.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-25T01:08:32.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-25T01:08:32.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:08:32.224 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:32.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:32.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:32.352 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:08:32.361 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-25T01:08:32.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:32.363 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-25T01:08:32.428 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-25T01:08:32.428 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:08:32.428 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T01:08:32.429 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T01:08:32.429 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T01:08:32.429 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:08:32.429 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:32.429 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:32.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T01:08:32.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T01:08:32.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T01:08:32.429 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:32.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:32.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:32.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:32.550 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:08:32.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:32.596 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-25T01:08:32.644 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:32.645 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:32.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:32.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T01:08:32.645 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:32.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:32.764 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T01:08:32.764 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:32.764 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:32.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T01:08:32.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:08:32.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T01:08:32.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T01:08:32.765 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T01:08:32.765 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:32.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:32.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:32.874 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:32.874 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:32.874 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-25T01:08:32.874 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:08:32.874 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:32.934 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T01:08:32.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:32.934 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:32.984 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T01:08:32.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:32.984 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:33.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:33.090 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:33.139 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:33.140 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:33.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:33.141 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:33.188 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:08:33.189 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T01:08:33.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:33.248 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:33.297 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:33.298 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:33.299 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:33.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:33.300 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:33.317 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:2024-04-25T01:08:33.325+0000 7f72be016580 -1 init, newargv = 0x5578284dd640 newargc=15 2024-04-25T01:08:33.317 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[33506]: starting ceph client 2024-04-25T01:08:33.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[33506]: starting fuse 2024-04-25T01:08:34.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:34.439 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:34.487 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:34.488 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:34.489 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:34.489 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:34.489 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:34.489 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:34.489 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:34.489 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:34.489 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:34.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:34.490 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:34.537 INFO:teuthology.orchestra.run.smithi033.stdout:45 2024-04-25T01:08:34.537 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T01:08:34.537 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:08:34.537 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-25T01:08:34.537 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-25T01:08:34.537 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-25T01:08:34.537 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-25T01:08:34.537 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:08:34.538 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-25T01:08:34.539 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-25T01:08:34.539 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-25T01:08:34.539 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:08:34.539 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:34.539 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.admin")) 2024-04-25T01:08:34.539 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-25T01:08:34.664 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.admin.33506.asok 2024-04-25T01:08:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:34.676 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33506.asok 2024-04-25T01:08:34.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:34.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33506.asok status 2024-04-25T01:08:34.837 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:08:34.837 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-25T01:08:34.837 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "admin", 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "33506", 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:34.838 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4434, 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4434 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:08:34.839 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3563027987 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3563027987 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4434 192.168.0.1:0/3563027987", 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/3563027987", 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 7, 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 25, 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:08:34.840 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-25T01:08:34.841 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-25T01:08:34.841 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:08:34.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:34.851 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-25T01:08:34.851 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:34.857 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-25T01:08:34.858 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:34.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:34.858 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:34.913 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T01:08:35.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T01:08:35.043 INFO:teuthology.orchestra.run.smithi033.stdout:# file: . 2024-04-25T01:08:35.043 INFO:teuthology.orchestra.run.smithi033.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T01:08:35.043 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:35.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:35.050 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:35.050 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:08:35.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:35.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4e273640 1 Processor -- start 2024-04-25T01:08:35.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4e273640 1 -- start start 2024-04-25T01:08:35.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4e273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a4806e1c0 0x7f8a4815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4e273640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8a4815ea20 0x7f8a480b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4e273640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a48147a20 con 0x7f8a48147640 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4e273640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8a481627e0 con 0x7f8a4815ea20 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4e273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4815f230 con 0x7f8a4806e1c0 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a47fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8a48147640 0x7f8a4806dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59388/0 (socket says 172.21.15.33:59388) 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a47fff640 1 -- 172.21.15.33:0/543143815 learned_addr learned my addr 172.21.15.33:0/543143815 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4c810640 1 --2- 172.21.15.33:0/543143815 >> v2:172.21.15.33:3300/0 conn(0x7f8a4815ea20 0x7f8a480b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a477fe640 1 --2- 172.21.15.33:0/543143815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a4806e1c0 0x7f8a4815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a46ffd640 1 -- 172.21.15.33:0/543143815 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 590039764 0 0) 0x7f8a48147a20 con 0x7f8a48147640 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a46ffd640 1 -- 172.21.15.33:0/543143815 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a30003530 con 0x7f8a48147640 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4c810640 1 -- 172.21.15.33:0/543143815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a4806e1c0 msgr2=0x7f8a4815e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4c810640 1 --2- 172.21.15.33:0/543143815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a4806e1c0 0x7f8a4815e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4c810640 1 -- 172.21.15.33:0/543143815 >> v1:172.21.15.33:6789/0 conn(0x7f8a48147640 legacy=0x7f8a4806dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.245+0000 7f8a4c810640 1 -- 172.21.15.33:0/543143815 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a48161a20 con 0x7f8a4815ea20 2024-04-25T01:08:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4c810640 1 --2- 172.21.15.33:0/543143815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a480b4840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009d90 tx=0x7f8a4001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8d3920cf11da49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a46ffd640 1 -- 172.21.15.33:0/543143815 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400b2070 con 0x7f8a4815ea20 2024-04-25T01:08:35.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a46ffd640 1 -- 172.21.15.33:0/543143815 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a400a5970 con 0x7f8a4815ea20 2024-04-25T01:08:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a46ffd640 1 -- 172.21.15.33:0/543143815 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400a5d20 con 0x7f8a4815ea20 2024-04-25T01:08:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- 172.21.15.33:0/543143815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 msgr2=0x7f8a480b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 --2- 172.21.15.33:0/543143815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a480b4840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009d90 tx=0x7f8a4001c8b0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- 172.21.15.33:0/543143815 shutdown_connections 2024-04-25T01:08:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 --2- 172.21.15.33:0/543143815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a480b4840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 --2- 172.21.15.33:0/543143815 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a4806e1c0 0x7f8a4815e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- 172.21.15.33:0/543143815 >> 172.21.15.33:0/543143815 conn(0x7f8a480b1810 msgr2=0x7f8a480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- 172.21.15.33:0/543143815 shutdown_connections 2024-04-25T01:08:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- 172.21.15.33:0/543143815 wait complete. 2024-04-25T01:08:35.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 Processor -- start 2024-04-25T01:08:35.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- start start 2024-04-25T01:08:35.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4806e1c0 0x7f8a48162a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a48147640 0x7f8a4815d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a48166d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8a480b5590 con 0x7f8a4815ea20 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4815eed0 con 0x7f8a4806e1c0 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4e273640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8a480b5710 con 0x7f8a48147640 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4c810640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a48166d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4c810640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a48166d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34470/0 (socket says 172.21.15.33:34470) 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a4c810640 1 -- 172.21.15.33:0/1616711621 learned_addr learned my addr 172.21.15.33:0/1616711621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.249+0000 7f8a477fe640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a48147640 0x7f8a4815d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a47fff640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4806e1c0 0x7f8a48162a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4c810640 1 -- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a48147640 msgr2=0x7f8a4815d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4c810640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a48147640 0x7f8a4815d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4c810640 1 -- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4806e1c0 msgr2=0x7f8a48162a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4c810640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4806e1c0 0x7f8a48162a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4c810640 1 -- 172.21.15.33:0/1616711621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a48167960 con 0x7f8a4815ea20 2024-04-25T01:08:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4c810640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a48166d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a400b0040 tx=0x7f8a40019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a44ff9640 1 -- 172.21.15.33:0/1616711621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400b2050 con 0x7f8a4815ea20 2024-04-25T01:08:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a44ff9640 1 -- 172.21.15.33:0/1616711621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a400be070 con 0x7f8a4815ea20 2024-04-25T01:08:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a48167680 con 0x7f8a4815ea20 2024-04-25T01:08:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a44ff9640 1 -- 172.21.15.33:0/1616711621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a400ada90 con 0x7f8a4815ea20 2024-04-25T01:08:35.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a48144fc0 con 0x7f8a4815ea20 2024-04-25T01:08:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a48069040 con 0x7f8a4815ea20 2024-04-25T01:08:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a44ff9640 1 -- 172.21.15.33:0/1616711621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8a400adc30 con 0x7f8a4815ea20 2024-04-25T01:08:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a44ff9640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8a340678f0 0x7f8a34069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a44ff9640 1 -- 172.21.15.33:0/1616711621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8a40121170 con 0x7f8a4815ea20 2024-04-25T01:08:35.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a47fff640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8a340678f0 0x7f8a34069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.253+0000 7f8a47fff640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8a340678f0 0x7f8a34069db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005890 tx=0x7f8a3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.257+0000 7f8a44ff9640 1 -- 172.21.15.33:0/1616711621 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8a40016560 con 0x7f8a4815ea20 2024-04-25T01:08:35.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.421+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a48162f40 con 0x7f8a4815ea20 2024-04-25T01:08:35.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a44ff9640 1 -- 172.21.15.33:0/1616711621 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f8a400ab340 con 0x7f8a4815ea20 2024-04-25T01:08:35.417 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:35.417 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:35.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8a340678f0 msgr2=0x7f8a34069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8a340678f0 0x7f8a34069db0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c005890 tx=0x7f8a3c009040 comp rx=0 tx=0).stop 2024-04-25T01:08:35.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 msgr2=0x7f8a48166d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a48166d10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8a400b0040 tx=0x7f8a40019650 comp rx=0 tx=0).stop 2024-04-25T01:08:35.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 shutdown_connections 2024-04-25T01:08:35.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8a340678f0 0x7f8a34069db0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a4815ea20 0x7f8a48166d10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8a48147640 0x7f8a4815d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 --2- 172.21.15.33:0/1616711621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a4806e1c0 0x7f8a48162a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 >> 172.21.15.33:0/1616711621 conn(0x7f8a480b1810 msgr2=0x7f8a4806d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:35.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 shutdown_connections 2024-04-25T01:08:35.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.425+0000 7f8a4e273640 1 -- 172.21.15.33:0/1616711621 wait complete. 2024-04-25T01:08:35.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:08:35.436 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:08:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.569+0000 7f83e46d3640 1 Processor -- start 2024-04-25T01:08:35.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.569+0000 7f83e46d3640 1 -- start start 2024-04-25T01:08:35.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e46d3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f83dc072180 0x7f83dc166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e46d3640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83dc160780 con 0x7f83dc06eb60 2024-04-25T01:08:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e46d3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83dc0cee10 con 0x7f83dc145660 2024-04-25T01:08:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e46d3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f83dc145a40 con 0x7f83dc072180 2024-04-25T01:08:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f83dc06eb60 0x7f83dc160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59392/0 (socket says 172.21.15.33:59392) 2024-04-25T01:08:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 -- 172.21.15.33:0/152269248 learned_addr learned my addr 172.21.15.33:0/152269248 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:35.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e2c70640 1 --2- 172.21.15.33:0/152269248 >> v2:172.21.15.138:3300/0 conn(0x7f83dc072180 0x7f83dc166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2622609780 0 0) 0x7f83dc160780 con 0x7f83dc06eb60 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83c8003530 con 0x7f83dc06eb60 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3926301705 0 0) 0x7f83c8003530 con 0x7f83dc06eb60 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 >> v2:172.21.15.138:3300/0 conn(0x7f83dc072180 msgr2=0x7f83dc166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 --2- 172.21.15.33:0/152269248 >> v2:172.21.15.138:3300/0 conn(0x7f83dc072180 0x7f83dc166c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 >> v1:172.21.15.138:6789/0 conn(0x7f83dc145660 legacy=0x7f83dc15f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc0cf0f0 con 0x7f83dc06eb60 2024-04-25T01:08:35.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f83cc001680 con 0x7f83dc06eb60 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 >> v1:172.21.15.33:6789/0 conn(0x7f83dc06eb60 legacy=0x7f83dc160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83c80039c0 0x7f83c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c80051d0 0x7f83c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83c800c040 con 0x7f83c8004810 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83c800c1f0 con 0x7f83c80039c0 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83c800c3f0 con 0x7f83c80051d0 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e246f640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83c80039c0 0x7f83c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e2c70640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c80051d0 0x7f83c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 -- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c80051d0 msgr2=0x7f83c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c80051d0 0x7f83c800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 -- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83c80039c0 msgr2=0x7f83c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83c80039c0 0x7f83c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e1c6e640 1 -- 172.21.15.33:0/152269248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c8003530 con 0x7f83c8004810 2024-04-25T01:08:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e246f640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83c80039c0 0x7f83c8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e2c70640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c80051d0 0x7f83c800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:35.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.573+0000 7f83e2c70640 1 -- 172.21.15.33:0/152269248 reap_dead start 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e1c6e640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c210 tx=0x7f83cc01f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b0c5c1f84ece6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc007d40 con 0x7f83c8004810 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc005b00 con 0x7f83c8004810 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e146d640 1 -- 172.21.15.33:0/152269248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc007070 con 0x7f83c8004810 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- 172.21.15.33:0/152269248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83c8004810 msgr2=0x7f83c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00c210 tx=0x7f83cc01f2f0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- 172.21.15.33:0/152269248 shutdown_connections 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 --2- 172.21.15.33:0/152269248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83c8004810 0x7f83c8004c90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- 172.21.15.33:0/152269248 >> 172.21.15.33:0/152269248 conn(0x7f83dc0b1810 msgr2=0x7f83dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- 172.21.15.33:0/152269248 shutdown_connections 2024-04-25T01:08:35.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- 172.21.15.33:0/152269248 wait complete. 2024-04-25T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 Processor -- start 2024-04-25T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- start start 2024-04-25T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c8004810 0x7f83dc1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83dc0711e0 0x7f83dc071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83dc14fc60 0x7f83dc1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc0cf6b0 con 0x7f83dc14fc60 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc070ce0 con 0x7f83dc0711e0 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e46d3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83dc06efa0 con 0x7f83c8004810 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83dc14fc60 0x7f83dc1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83dc14fc60 0x7f83dc1500e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34492/0 (socket says 172.21.15.33:34492) 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 -- 172.21.15.33:0/848229251 learned_addr learned my addr 172.21.15.33:0/848229251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e1c6e640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83dc0711e0 0x7f83dc071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e246f640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c8004810 0x7f83dc1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 -- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c8004810 msgr2=0x7f83dc1401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c8004810 0x7f83dc1401e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 -- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83dc0711e0 msgr2=0x7f83dc071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83dc0711e0 0x7f83dc071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e2c70640 1 -- 172.21.15.33:0/848229251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83dc13f140 con 0x7f83dc14fc60 2024-04-25T01:08:35.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.577+0000 7f83e1c6e640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83dc0711e0 0x7f83dc071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83e2c70640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83dc14fc60 0x7f83dc1500e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cd350 tx=0x7f83d400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83d2ffd640 1 -- 172.21.15.33:0/848229251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4016620 con 0x7f83dc14fc60 2024-04-25T01:08:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83d2ffd640 1 -- 172.21.15.33:0/848229251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d401b070 con 0x7f83dc14fc60 2024-04-25T01:08:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83dc13f420 con 0x7f83dc14fc60 2024-04-25T01:08:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83d2ffd640 1 -- 172.21.15.33:0/848229251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40169c0 con 0x7f83dc14fc60 2024-04-25T01:08:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83dc13f9e0 con 0x7f83dc14fc60 2024-04-25T01:08:35.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83dc069040 con 0x7f83dc14fc60 2024-04-25T01:08:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83d2ffd640 1 -- 172.21.15.33:0/848229251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83d4016b60 con 0x7f83dc14fc60 2024-04-25T01:08:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83d2ffd640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f83c00678c0 0x7f83c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83d2ffd640 1 -- 172.21.15.33:0/848229251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f83d4086ea0 con 0x7f83dc14fc60 2024-04-25T01:08:35.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83e246f640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f83c00678c0 0x7f83c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.581+0000 7f83e246f640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f83c00678c0 0x7f83c0069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f83d8000c10 tx=0x7f83d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.585+0000 7f83d2ffd640 1 -- 172.21.15.33:0/848229251 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f83d4025020 con 0x7f83dc14fc60 2024-04-25T01:08:35.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83dc0b1580 con 0x7f83dc14fc60 2024-04-25T01:08:35.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83d2ffd640 1 -- 172.21.15.33:0/848229251 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f83d40576a0 con 0x7f83dc14fc60 2024-04-25T01:08:35.741 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:08:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f83c00678c0 msgr2=0x7f83c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f83c00678c0 0x7f83c0069d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f83d8000c10 tx=0x7f83d8009040 comp rx=0 tx=0).stop 2024-04-25T01:08:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83dc14fc60 msgr2=0x7f83dc1500e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83dc14fc60 0x7f83dc1500e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0cd350 tx=0x7f83d400f520 comp rx=0 tx=0).stop 2024-04-25T01:08:35.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 shutdown_connections 2024-04-25T01:08:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f83c00678c0 0x7f83c0069d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83dc14fc60 0x7f83dc1500e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83dc0711e0 0x7f83dc071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 --2- 172.21.15.33:0/848229251 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83c8004810 0x7f83dc1401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.749+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 >> 172.21.15.33:0/848229251 conn(0x7f83dc0b1810 msgr2=0x7f83dc0ad9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.753+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 shutdown_connections 2024-04-25T01:08:35.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.753+0000 7f83e46d3640 1 -- 172.21.15.33:0/848229251 wait complete. 2024-04-25T01:08:35.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:08:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.893+0000 7f19fbf73640 1 Processor -- start 2024-04-25T01:08:35.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.893+0000 7f19fbf73640 1 -- start start 2024-04-25T01:08:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19fbf73640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f19f4145660 0x7f19f406eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19fbf73640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 0x7f19f4071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19fbf73640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f40cee10 con 0x7f19f415f960 2024-04-25T01:08:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19fbf73640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f19f406f0a0 con 0x7f19f4145660 2024-04-25T01:08:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19fbf73640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19f4145a40 con 0x7f19f41604b0 2024-04-25T01:08:35.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f950e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f19f415f960 0x7f19f415fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59394/0 (socket says 172.21.15.33:59394) 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f950e640 1 -- 172.21.15.33:0/3886218524 learned_addr learned my addr 172.21.15.33:0/3886218524 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/3886218524 >> v2:172.21.15.138:3300/0 conn(0x7f19f4145660 0x7f19f406eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19fa510640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 0x7f19f4071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 942315387 0 0) 0x7f19f40cee10 con 0x7f19f415f960 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19e0003530 con 0x7f19f415f960 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2373387301 0 0) 0x7f19e0003530 con 0x7f19f415f960 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 msgr2=0x7f19f4071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 0x7f19f4071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 >> v2:172.21.15.138:3300/0 conn(0x7f19f4145660 msgr2=0x7f19f406eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 --2- 172.21.15.33:0/3886218524 >> v2:172.21.15.138:3300/0 conn(0x7f19f4145660 0x7f19f406eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f40cf0f0 con 0x7f19f415f960 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f19e4001680 con 0x7f19f415f960 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/3886218524 >> v2:172.21.15.138:3300/0 conn(0x7f19f4145660 0x7f19f406eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 >> v1:172.21.15.33:6789/0 conn(0x7f19f415f960 legacy=0x7f19f415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 0x7f19e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19e0004d50 0x7f19e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19f406f0a0 con 0x7f19e00039c0 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19e000b9d0 con 0x7f19e0004d50 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19e000bb50 con 0x7f19f41604b0 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f950e640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 0x7f19e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19fa510640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19e0004d50 0x7f19e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 -- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 msgr2=0x7f19e0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41604b0 0x7f19e0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 -- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19e0004d50 msgr2=0x7f19e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19e0004d50 0x7f19e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.897+0000 7f19f9d0f640 1 -- 172.21.15.33:0/3886218524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e0003530 con 0x7f19e00039c0 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19e0003de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e350 tx=0x7f19ec01ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddd87a87e76ddae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0b2070 con 0x7f19e00039c0 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec0a5d60 con 0x7f19e00039c0 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19f8d0d640 1 -- 172.21.15.33:0/3886218524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec009de0 con 0x7f19e00039c0 2024-04-25T01:08:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- 172.21.15.33:0/3886218524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 msgr2=0x7f19e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 --2- 172.21.15.33:0/3886218524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19e0003de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00e350 tx=0x7f19ec01ce80 comp rx=0 tx=0).stop 2024-04-25T01:08:35.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fa510640 1 -- 172.21.15.33:0/3886218524 reap_dead start 2024-04-25T01:08:35.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- 172.21.15.33:0/3886218524 shutdown_connections 2024-04-25T01:08:35.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- 172.21.15.33:0/3886218524 >> 172.21.15.33:0/3886218524 conn(0x7f19f40b1810 msgr2=0x7f19f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:35.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- 172.21.15.33:0/3886218524 shutdown_connections 2024-04-25T01:08:35.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- 172.21.15.33:0/3886218524 wait complete. 2024-04-25T01:08:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 Processor -- start 2024-04-25T01:08:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- start start 2024-04-25T01:08:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19f413fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41572a0 0x7f19f4144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4158140 0x7f19f41585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4071d70 con 0x7f19e00039c0 2024-04-25T01:08:35.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f19f4071bf0 con 0x7f19f4158140 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19fbf73640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19f4071ef0 con 0x7f19f41572a0 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19f9d0f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19f413fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19f9d0f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19f413fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34500/0 (socket says 172.21.15.33:34500) 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.901+0000 7f19f9d0f640 1 -- 172.21.15.33:0/324352156 learned_addr learned my addr 172.21.15.33:0/324352156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f950e640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41572a0 0x7f19f4144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19fa510640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4158140 0x7f19f41585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f9d0f640 1 -- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41572a0 msgr2=0x7f19f4144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41572a0 0x7f19f4144990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f9d0f640 1 -- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4158140 msgr2=0x7f19f41585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4158140 0x7f19f41585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f9d0f640 1 -- 172.21.15.33:0/324352156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f4144ed0 con 0x7f19e00039c0 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19fa510640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4158140 0x7f19f41585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:35.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f9d0f640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19f413fc30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b0040 tx=0x7f19ec009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19ea7fc640 1 -- 172.21.15.33:0/324352156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0ba020 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19ea7fc640 1 -- 172.21.15.33:0/324352156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec0adb40 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f413eb90 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19ea7fc640 1 -- 172.21.15.33:0/324352156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0b2040 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f413f150 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f4157680 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19ea7fc640 1 -- 172.21.15.33:0/324352156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f19ec0bd020 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19ea7fc640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19d80678f0 0x7f19d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19ea7fc640 1 -- 172.21.15.33:0/324352156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f19ec120680 con 0x7f19e00039c0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f950e640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19d80678f0 0x7f19d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.905+0000 7f19f950e640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19d80678f0 0x7f19d8069db0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19e4001060 tx=0x7f19e4000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:35.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:35.909+0000 7f19ea7fc640 1 -- 172.21.15.33:0/324352156 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f19f4144ed0 con 0x7f19e00039c0 2024-04-25T01:08:36.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19f4069040 con 0x7f19e00039c0 2024-04-25T01:08:36.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19ea7fc640 1 -- 172.21.15.33:0/324352156 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f19ec016b90 con 0x7f19e00039c0 2024-04-25T01:08:36.066 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19d80678f0 msgr2=0x7f19d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19d80678f0 0x7f19d8069db0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19e4001060 tx=0x7f19e4000ff0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 msgr2=0x7f19f413fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19f413fc30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b0040 tx=0x7f19ec009750 comp rx=0 tx=0).stop 2024-04-25T01:08:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 shutdown_connections 2024-04-25T01:08:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19d80678f0 0x7f19d8069db0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19f4158140 0x7f19f41585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f19f41572a0 0x7f19f4144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 --2- 172.21.15.33:0/324352156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19e00039c0 0x7f19f413fc30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 >> 172.21.15.33:0/324352156 conn(0x7f19f40b1810 msgr2=0x7f19f40af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.073+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 shutdown_connections 2024-04-25T01:08:36.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.077+0000 7f19fbf73640 1 -- 172.21.15.33:0/324352156 wait complete. 2024-04-25T01:08:36.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:08:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf202b2640 1 Processor -- start 2024-04-25T01:08:36.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf202b2640 1 -- start start 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf202b2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 0x7fbf1806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf202b2640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf180cee10 con 0x7fbf1815f960 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf202b2640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf180cf0f0 con 0x7fbf181604b0 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf202b2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1806f0a0 con 0x7fbf18145660 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d84d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbf1815f960 0x7fbf1815fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59410/0 (socket says 172.21.15.33:59410) 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d84d640 1 -- 172.21.15.33:0/3484775169 learned_addr learned my addr 172.21.15.33:0/3484775169 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 0x7fbf1806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3128839891 0 0) 0x7fbf180cee10 con 0x7fbf1815f960 2024-04-25T01:08:36.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbf04003530 con 0x7fbf1815f960 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1980090101 0 0) 0x7fbf04003530 con 0x7fbf1815f960 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 msgr2=0x7fbf1806eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 0x7fbf1806eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 >> v1:172.21.15.138:6789/0 conn(0x7fbf181604b0 legacy=0x7fbf18071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf180cf3d0 con 0x7fbf1815f960 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fbf08001680 con 0x7fbf1815f960 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 >> v1:172.21.15.33:6789/0 conn(0x7fbf1815f960 legacy=0x7fbf1815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf040039f0 0x7fbf04003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 0x7fbf04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf04004d50 0x7fbf040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf1806f0a0 con 0x7fbf040039f0 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0400ba80 con 0x7fbf04004d50 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbf0400bc80 con 0x7fbf18145660 2024-04-25T01:08:36.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf040039f0 0x7fbf04003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1d84d640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 0x7fbf04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e04e640 1 -- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 msgr2=0x7fbf04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf18145660 0x7fbf04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e84f640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf04004d50 0x7fbf040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e04e640 1 -- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf04004d50 msgr2=0x7fbf040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf04004d50 0x7fbf040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e04e640 1 -- 172.21.15.33:0/3484775169 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf04003530 con 0x7fbf040039f0 2024-04-25T01:08:36.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.225+0000 7fbf1e84f640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf04004d50 0x7fbf040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf040039f0 0x7fbf04003e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009a90 tx=0x7fbf1401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab3a095a3198a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf14009690 con 0x7fbf040039f0 2024-04-25T01:08:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf140a6410 con 0x7fbf040039f0 2024-04-25T01:08:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf1d04c640 1 -- 172.21.15.33:0/3484775169 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf140a67e0 con 0x7fbf040039f0 2024-04-25T01:08:36.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 -- 172.21.15.33:0/3484775169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf040039f0 msgr2=0x7fbf04003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 --2- 172.21.15.33:0/3484775169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf040039f0 0x7fbf04003e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbf14009a90 tx=0x7fbf1401ca30 comp rx=0 tx=0).stop 2024-04-25T01:08:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf1e84f640 1 -- 172.21.15.33:0/3484775169 reap_dead start 2024-04-25T01:08:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 -- 172.21.15.33:0/3484775169 shutdown_connections 2024-04-25T01:08:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 -- 172.21.15.33:0/3484775169 >> 172.21.15.33:0/3484775169 conn(0x7fbf180b1810 msgr2=0x7fbf180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 -- 172.21.15.33:0/3484775169 shutdown_connections 2024-04-25T01:08:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 -- 172.21.15.33:0/3484775169 wait complete. 2024-04-25T01:08:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 Processor -- start 2024-04-25T01:08:36.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.229+0000 7fbf202b2640 1 -- start start 2024-04-25T01:08:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf040039f0 0x7fbf1813ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf1815b590 0x7fbf18137860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf181f72d0 0x7fbf181f7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf18072460 con 0x7fbf181f72d0 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf180722e0 con 0x7fbf1815b590 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbf18070ce0 con 0x7fbf040039f0 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf181f72d0 0x7fbf181f7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf181f72d0 0x7fbf181f7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34508/0 (socket says 172.21.15.33:34508) 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 -- 172.21.15.33:0/2705423983 learned_addr learned my addr 172.21.15.33:0/2705423983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf040039f0 0x7fbf1813ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1d84d640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf1815b590 0x7fbf18137860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 -- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf040039f0 msgr2=0x7fbf1813ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf040039f0 0x7fbf1813ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 -- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf1815b590 msgr2=0x7fbf18137860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf1815b590 0x7fbf18137860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 -- 172.21.15.33:0/2705423983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf18137da0 con 0x7fbf181f72d0 2024-04-25T01:08:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e84f640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf181f72d0 0x7fbf181f7750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf1806cbc0 tx=0x7fbf1000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf0effd640 1 -- 172.21.15.33:0/2705423983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10006d60 con 0x7fbf181f72d0 2024-04-25T01:08:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf0effd640 1 -- 172.21.15.33:0/2705423983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf10003980 con 0x7fbf181f72d0 2024-04-25T01:08:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf180b34d0 con 0x7fbf181f72d0 2024-04-25T01:08:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf0effd640 1 -- 172.21.15.33:0/2705423983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf10003d70 con 0x7fbf181f72d0 2024-04-25T01:08:36.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf180b3a90 con 0x7fbf181f72d0 2024-04-25T01:08:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf18069040 con 0x7fbf181f72d0 2024-04-25T01:08:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf0effd640 1 -- 172.21.15.33:0/2705423983 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf1001b020 con 0x7fbf181f72d0 2024-04-25T01:08:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf0effd640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbefc0678c0 0x7fbefc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf0effd640 1 -- 172.21.15.33:0/2705423983 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbf10085c00 con 0x7fbf181f72d0 2024-04-25T01:08:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.233+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbefc0678c0 0x7fbefc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.237+0000 7fbf1e04e640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbefc0678c0 0x7fbefc069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf140ad000 tx=0x7fbf140b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.237+0000 7fbf0effd640 1 -- 172.21.15.33:0/2705423983 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbf1008c080 con 0x7fbf181f72d0 2024-04-25T01:08:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbf18160050 con 0x7fbf181f72d0 2024-04-25T01:08:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf0effd640 1 -- 172.21.15.33:0/2705423983 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fbf10056490 con 0x7fbf181f72d0 2024-04-25T01:08:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbefc0678c0 msgr2=0x7fbefc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbefc0678c0 0x7fbefc069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbf140ad000 tx=0x7fbf140b2040 comp rx=0 tx=0).stop 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf181f72d0 msgr2=0x7fbf181f7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf181f72d0 0x7fbf181f7750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf1806cbc0 tx=0x7fbf1000dc00 comp rx=0 tx=0).stop 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 shutdown_connections 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbefc0678c0 0x7fbefc069d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf181f72d0 0x7fbf181f7750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf1815b590 0x7fbf18137860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 --2- 172.21.15.33:0/2705423983 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbf040039f0 0x7fbf1813ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 >> 172.21.15.33:0/2705423983 conn(0x7fbf180b1810 msgr2=0x7fbf180afcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 shutdown_connections 2024-04-25T01:08:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.401+0000 7fbf202b2640 1 -- 172.21.15.33:0/2705423983 wait complete. 2024-04-25T01:08:36.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:08:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f9364b63640 1 Processor -- start 2024-04-25T01:08:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f9364b63640 1 -- start start 2024-04-25T01:08:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f9364b63640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f93600d2ac0 0x7f9360168d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f9364b63640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600b6cc0 con 0x7f936015e0d0 2024-04-25T01:08:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f9364b63640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93600b6f30 con 0x7f93600d1f60 2024-04-25T01:08:36.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f9364b63640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f93600cfb00 con 0x7f93600d2ac0 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935e59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f936015e0d0 0x7f93600cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59414/0 (socket says 172.21.15.33:59414) 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935e59c640 1 -- 172.21.15.33:0/3267355322 learned_addr learned my addr 172.21.15.33:0/3267355322 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935ed9d640 1 --2- 172.21.15.33:0/3267355322 >> v2:172.21.15.138:3300/0 conn(0x7f93600d2ac0 0x7f9360168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935dd9b640 1 -- 172.21.15.33:0/3267355322 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2766023348 0 0) 0x7f93600b6cc0 con 0x7f936015e0d0 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935dd9b640 1 -- 172.21.15.33:0/3267355322 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f934c003530 con 0x7f936015e0d0 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935ed9d640 1 -- 172.21.15.33:0/3267355322 >> v1:172.21.15.138:6789/0 conn(0x7f93600d1f60 legacy=0x7f93600d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935ed9d640 1 -- 172.21.15.33:0/3267355322 >> v1:172.21.15.33:6789/0 conn(0x7f936015e0d0 legacy=0x7f93600cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.545+0000 7f935ed9d640 1 -- 172.21.15.33:0/3267355322 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93600b7210 con 0x7f93600d2ac0 2024-04-25T01:08:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f935ed9d640 1 --2- 172.21.15.33:0/3267355322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f9360168d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9358009d90 tx=0x7f935801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc01bb27fb4903b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f935dd9b640 1 -- 172.21.15.33:0/3267355322 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580b2070 con 0x7f93600d2ac0 2024-04-25T01:08:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f935dd9b640 1 -- 172.21.15.33:0/3267355322 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93580a5850 con 0x7f93600d2ac0 2024-04-25T01:08:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f935dd9b640 1 -- 172.21.15.33:0/3267355322 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93580a5c00 con 0x7f93600d2ac0 2024-04-25T01:08:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- 172.21.15.33:0/3267355322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 msgr2=0x7f9360168d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 --2- 172.21.15.33:0/3267355322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f9360168d00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9358009d90 tx=0x7f935801c970 comp rx=0 tx=0).stop 2024-04-25T01:08:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- 172.21.15.33:0/3267355322 shutdown_connections 2024-04-25T01:08:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 --2- 172.21.15.33:0/3267355322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f9360168d00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- 172.21.15.33:0/3267355322 >> 172.21.15.33:0/3267355322 conn(0x7f93600d3dd0 msgr2=0x7f93600d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- 172.21.15.33:0/3267355322 shutdown_connections 2024-04-25T01:08:36.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- 172.21.15.33:0/3267355322 wait complete. 2024-04-25T01:08:36.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 Processor -- start 2024-04-25T01:08:36.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- start start 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93600d1f60 0x7f9360154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f93601607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936015e0d0 0x7f93600caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f936016bdf0 con 0x7f93600d1f60 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f936016bc70 con 0x7f93600d2ac0 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.549+0000 7f9364b63640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f936016bf70 con 0x7f936015e0d0 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93600d1f60 0x7f9360154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93600d1f60 0x7f9360154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34514/0 (socket says 172.21.15.33:34514) 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 -- 172.21.15.33:0/3304226952 learned_addr learned my addr 172.21.15.33:0/3304226952 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935ed9d640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936015e0d0 0x7f93600caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9357fff640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f93601607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 -- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936015e0d0 msgr2=0x7f93600caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936015e0d0 0x7f93600caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 -- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 msgr2=0x7f93601607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f93601607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 -- 172.21.15.33:0/3304226952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9360155550 con 0x7f93600d1f60 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935ed9d640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936015e0d0 0x7f93600caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9357fff640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f93601607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f935e59c640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93600d1f60 0x7f9360154f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f935000c4d0 tx=0x7f935000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9356ffd640 1 -- 172.21.15.33:0/3304226952 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350017070 con 0x7f93600d1f60 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9356ffd640 1 -- 172.21.15.33:0/3304226952 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9350005cf0 con 0x7f93600d1f60 2024-04-25T01:08:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9356ffd640 1 -- 172.21.15.33:0/3304226952 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350007270 con 0x7f93600d1f60 2024-04-25T01:08:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9360155830 con 0x7f93600d1f60 2024-04-25T01:08:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9360155df0 con 0x7f93600d1f60 2024-04-25T01:08:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9360069040 con 0x7f93600d1f60 2024-04-25T01:08:36.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9356ffd640 1 -- 172.21.15.33:0/3304226952 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f935001a070 con 0x7f93600d1f60 2024-04-25T01:08:36.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9356ffd640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93400678c0 0x7f9340069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9356ffd640 1 -- 172.21.15.33:0/3304226952 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f93500852a0 con 0x7f93600d1f60 2024-04-25T01:08:36.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9357fff640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93400678c0 0x7f9340069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.553+0000 7f9357fff640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93400678c0 0x7f9340069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93600cdea0 tx=0x7f9348009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.557+0000 7f9356ffd640 1 -- 172.21.15.33:0/3304226952 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f93500554e0 con 0x7f93600d1f60 2024-04-25T01:08:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f93601618d0 con 0x7f93600d1f60 2024-04-25T01:08:36.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9356ffd640 1 -- 172.21.15.33:0/3304226952 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f93500597e0 con 0x7f93600d1f60 2024-04-25T01:08:36.717 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93400678c0 msgr2=0x7f9340069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93400678c0 0x7f9340069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93600cdea0 tx=0x7f9348009040 comp rx=0 tx=0).stop 2024-04-25T01:08:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93600d1f60 msgr2=0x7f9360154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93600d1f60 0x7f9360154f70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f935000c4d0 tx=0x7f935000c990 comp rx=0 tx=0).stop 2024-04-25T01:08:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 shutdown_connections 2024-04-25T01:08:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93400678c0 0x7f9340069d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f936015e0d0 0x7f93600caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93600d2ac0 0x7f93601607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 --2- 172.21.15.33:0/3304226952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f93600d1f60 0x7f9360154f70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 >> 172.21.15.33:0/3304226952 conn(0x7f93600d3dd0 msgr2=0x7f93600d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 shutdown_connections 2024-04-25T01:08:36.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.725+0000 7f9364b63640 1 -- 172.21.15.33:0/3304226952 wait complete. 2024-04-25T01:08:36.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:08:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 Processor -- start 2024-04-25T01:08:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 -- start start 2024-04-25T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efc50145470 0x7efc5015c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc500cf0e0 0x7efc500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf570 0x7efc500b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7efc5015f750 con 0x7efc500cf0e0 2024-04-25T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efc5015f9a0 con 0x7efc50145470 2024-04-25T01:08:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc583df640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efc5015fc80 con 0x7efc500cf570 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc500cf0e0 0x7efc500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efc500cf0e0 0x7efc500b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34530/0 (socket says 172.21.15.33:34530) 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 -- 172.21.15.33:0/3713879797 learned_addr learned my addr 172.21.15.33:0/3713879797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5617b640 1 --2- 172.21.15.33:0/3713879797 >> v2:172.21.15.138:3300/0 conn(0x7efc50145470 0x7efc5015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5697c640 1 --2- 172.21.15.33:0/3713879797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf570 0x7efc500b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 -- 172.21.15.33:0/3713879797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf570 msgr2=0x7efc500b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 --2- 172.21.15.33:0/3713879797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf570 0x7efc500b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 -- 172.21.15.33:0/3713879797 >> v2:172.21.15.138:3300/0 conn(0x7efc50145470 msgr2=0x7efc5015c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 --2- 172.21.15.33:0/3713879797 >> v2:172.21.15.138:3300/0 conn(0x7efc50145470 0x7efc5015c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.873+0000 7efc5597a640 1 -- 172.21.15.33:0/3713879797 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5015ff60 con 0x7efc500cf0e0 2024-04-25T01:08:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc5597a640 1 --2- 172.21.15.33:0/3713879797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc500cf0e0 0x7efc500b4e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc5006f1b0 tx=0x7efc4001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88e12274c60b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc55179640 1 -- 172.21.15.33:0/3713879797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400aa600 con 0x7efc500cf0e0 2024-04-25T01:08:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc55179640 1 -- 172.21.15.33:0/3713879797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc400af070 con 0x7efc500cf0e0 2024-04-25T01:08:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc55179640 1 -- 172.21.15.33:0/3713879797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400aa9c0 con 0x7efc500cf0e0 2024-04-25T01:08:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 -- 172.21.15.33:0/3713879797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc500cf0e0 msgr2=0x7efc500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 --2- 172.21.15.33:0/3713879797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc500cf0e0 0x7efc500b4e20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efc5006f1b0 tx=0x7efc4001c520 comp rx=0 tx=0).stop 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 -- 172.21.15.33:0/3713879797 shutdown_connections 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 --2- 172.21.15.33:0/3713879797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf570 0x7efc500b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 --2- 172.21.15.33:0/3713879797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc500cf0e0 0x7efc500b4e20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 --2- 172.21.15.33:0/3713879797 >> v2:172.21.15.138:3300/0 conn(0x7efc50145470 0x7efc5015c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 -- 172.21.15.33:0/3713879797 >> 172.21.15.33:0/3713879797 conn(0x7efc500b1810 msgr2=0x7efc500b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 -- 172.21.15.33:0/3713879797 shutdown_connections 2024-04-25T01:08:36.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 -- 172.21.15.33:0/3713879797 wait complete. 2024-04-25T01:08:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 Processor -- start 2024-04-25T01:08:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.877+0000 7efc583df640 1 -- start start 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf0e0 0x7efc5015aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc500cf570 0x7efc500ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc50145470 0x7efc500cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc50072520 con 0x7efc50145470 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efc500723a0 con 0x7efc500cf570 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efc500726a0 con 0x7efc500cf0e0 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc50145470 0x7efc500cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc50145470 0x7efc500cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34546/0 (socket says 172.21.15.33:34546) 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 -- 172.21.15.33:0/2731018872 learned_addr learned my addr 172.21.15.33:0/2731018872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5617b640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf0e0 0x7efc5015aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5597a640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc500cf570 0x7efc500ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 -- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf0e0 msgr2=0x7efc5015aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf0e0 0x7efc5015aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 -- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc500cf570 msgr2=0x7efc500ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc500cf570 0x7efc500ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 -- 172.21.15.33:0/2731018872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc500aa420 con 0x7efc50145470 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5597a640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc500cf570 0x7efc500ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5697c640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc50145470 0x7efc500cd240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc48007e60 tx=0x7efc4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc46ffd640 1 -- 172.21.15.33:0/2731018872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48014620 con 0x7efc50145470 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc46ffd640 1 -- 172.21.15.33:0/2731018872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc48019070 con 0x7efc50145470 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc500aa700 con 0x7efc50145470 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc46ffd640 1 -- 172.21.15.33:0/2731018872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48014a20 con 0x7efc50145470 2024-04-25T01:08:36.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc500aacc0 con 0x7efc50145470 2024-04-25T01:08:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc50069040 con 0x7efc50145470 2024-04-25T01:08:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc46ffd640 1 -- 172.21.15.33:0/2731018872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efc48014bc0 con 0x7efc50145470 2024-04-25T01:08:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc46ffd640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efc340678c0 0x7efc34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc46ffd640 1 -- 172.21.15.33:0/2731018872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efc480867b0 con 0x7efc50145470 2024-04-25T01:08:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.881+0000 7efc5617b640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efc340678c0 0x7efc34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.885+0000 7efc5617b640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efc340678c0 0x7efc34069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b930 tx=0x7efc4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:36.885+0000 7efc46ffd640 1 -- 172.21.15.33:0/2731018872 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efc48057040 con 0x7efc50145470 2024-04-25T01:08:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.065+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efc50071420 con 0x7efc50145470 2024-04-25T01:08:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc46ffd640 1 -- 172.21.15.33:0/2731018872 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7efc48056940 con 0x7efc50145470 2024-04-25T01:08:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:08:37.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efc340678c0 msgr2=0x7efc34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efc340678c0 0x7efc34069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efc4c00b930 tx=0x7efc4c008040 comp rx=0 tx=0).stop 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc50145470 msgr2=0x7efc500cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc50145470 0x7efc500cd240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efc48007e60 tx=0x7efc4800e4b0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 shutdown_connections 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efc340678c0 0x7efc34069d80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc50145470 0x7efc500cd240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc500cf570 0x7efc500ccd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 --2- 172.21.15.33:0/2731018872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efc500cf0e0 0x7efc5015aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 >> 172.21.15.33:0/2731018872 conn(0x7efc500b1810 msgr2=0x7efc50141d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 shutdown_connections 2024-04-25T01:08:37.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.069+0000 7efc583df640 1 -- 172.21.15.33:0/2731018872 wait complete. 2024-04-25T01:08:37.079 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:08:37.079 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:08:37.084 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.261+0000 7fd6f7869640 1 Processor -- start 2024-04-25T01:08:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.261+0000 7fd6f7869640 1 -- start start 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f7869640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd6f015dfa0 0x7fd6f00d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f7869640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6f00d4950 0x7fd6f00d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f7869640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6f00d25b0 con 0x7fd6f00d11a0 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f7869640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f00d15b0 con 0x7fd6f00d4950 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f7869640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f00d2890 con 0x7fd6f015dfa0 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6f00d4950 0x7fd6f00d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd6f00d4950 0x7fd6f00d0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34556/0 (socket says 172.21.15.33:34556) 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 -- 172.21.15.33:0/2477880928 learned_addr learned my addr 172.21.15.33:0/2477880928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 -- 172.21.15.33:0/2477880928 >> v2:172.21.15.138:3300/0 conn(0x7fd6f015dfa0 msgr2=0x7fd6f00d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 --2- 172.21.15.33:0/2477880928 >> v2:172.21.15.138:3300/0 conn(0x7fd6f015dfa0 0x7fd6f00d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 -- 172.21.15.33:0/2477880928 >> v1:172.21.15.138:6789/0 conn(0x7fd6f00d11a0 legacy=0x7fd6f0161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 -- 172.21.15.33:0/2477880928 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f00d2b70 con 0x7fd6f00d4950 2024-04-25T01:08:37.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6f4e04640 1 --2- 172.21.15.33:0/2477880928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f00d0c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0067a60 tx=0x7fd6e001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99d04d240de8c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6e7fff640 1 -- 172.21.15.33:0/2477880928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b2070 con 0x7fd6f00d4950 2024-04-25T01:08:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6e7fff640 1 -- 172.21.15.33:0/2477880928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e00a9070 con 0x7fd6f00d4950 2024-04-25T01:08:37.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.265+0000 7fd6e7fff640 1 -- 172.21.15.33:0/2477880928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00ad880 con 0x7fd6f00d4950 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- 172.21.15.33:0/2477880928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 msgr2=0x7fd6f00d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 --2- 172.21.15.33:0/2477880928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f00d0c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0067a60 tx=0x7fd6e001c450 comp rx=0 tx=0).stop 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- 172.21.15.33:0/2477880928 shutdown_connections 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 --2- 172.21.15.33:0/2477880928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f00d0c60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 --2- 172.21.15.33:0/2477880928 >> v2:172.21.15.138:3300/0 conn(0x7fd6f015dfa0 0x7fd6f00d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- 172.21.15.33:0/2477880928 >> 172.21.15.33:0/2477880928 conn(0x7fd6f00d3e30 msgr2=0x7fd6f00d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- 172.21.15.33:0/2477880928 shutdown_connections 2024-04-25T01:08:37.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- 172.21.15.33:0/2477880928 wait complete. 2024-04-25T01:08:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 Processor -- start 2024-04-25T01:08:37.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- start start 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f00d11a0 0x7fd6f0150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f015d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6f015dfa0 0x7fd6f015b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f00b0bb0 con 0x7fd6f00d4950 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f00b0a30 con 0x7fd6f00d11a0 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f016aee0 con 0x7fd6f015dfa0 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f015d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f015d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34560/0 (socket says 172.21.15.33:34560) 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 -- 172.21.15.33:0/3496722067 learned_addr learned my addr 172.21.15.33:0/3496722067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f5605640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f00d11a0 0x7fd6f0150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f5e06640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6f015dfa0 0x7fd6f015b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 -- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6f015dfa0 msgr2=0x7fd6f015b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6f015dfa0 0x7fd6f015b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 -- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f00d11a0 msgr2=0x7fd6f0150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f00d11a0 0x7fd6f0150740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 -- 172.21.15.33:0/3496722067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f00b4140 con 0x7fd6f00d4950 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f5605640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f00d11a0 0x7fd6f0150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f4e04640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f015d2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6e001a080 tx=0x7fd6e001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6e5ffb640 1 -- 172.21.15.33:0/3496722067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00bc020 con 0x7fd6f00d4950 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6e5ffb640 1 -- 172.21.15.33:0/3496722067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e00b8420 con 0x7fd6f00d4950 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f00b4420 con 0x7fd6f00d4950 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6e5ffb640 1 -- 172.21.15.33:0/3496722067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b2040 con 0x7fd6f00d4950 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.269+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f01512a0 con 0x7fd6f00d4950 2024-04-25T01:08:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.273+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f0069040 con 0x7fd6f00d4950 2024-04-25T01:08:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.273+0000 7fd6e5ffb640 1 -- 172.21.15.33:0/3496722067 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd6e00bf020 con 0x7fd6f00d4950 2024-04-25T01:08:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.273+0000 7fd6e5ffb640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6d40678c0 0x7fd6d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.273+0000 7fd6e5ffb640 1 -- 172.21.15.33:0/3496722067 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6e01206a0 con 0x7fd6f00d4950 2024-04-25T01:08:37.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.273+0000 7fd6f5605640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6d40678c0 0x7fd6d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.273+0000 7fd6f5605640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6d40678c0 0x7fd6d4069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00d4230 tx=0x7fd6ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.277+0000 7fd6e5ffb640 1 -- 172.21.15.33:0/3496722067 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd6e00c6030 con 0x7fd6f00d4950 2024-04-25T01:08:37.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.441+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6f008dbe0 con 0x7fd6f00d4950 2024-04-25T01:08:37.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.441+0000 7fd6e5ffb640 1 -- 172.21.15.33:0/3496722067 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fd6e00160d0 con 0x7fd6f00d4950 2024-04-25T01:08:37.436 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:37.436 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6d40678c0 msgr2=0x7fd6d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6d40678c0 0x7fd6d4069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00d4230 tx=0x7fd6ec008040 comp rx=0 tx=0).stop 2024-04-25T01:08:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 msgr2=0x7fd6f015d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f015d2e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd6e001a080 tx=0x7fd6e001ac90 comp rx=0 tx=0).stop 2024-04-25T01:08:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 shutdown_connections 2024-04-25T01:08:37.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6d40678c0 0x7fd6d4069d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6f015dfa0 0x7fd6f015b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6f00d4950 0x7fd6f015d2e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 --2- 172.21.15.33:0/3496722067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6f00d11a0 0x7fd6f0150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 >> 172.21.15.33:0/3496722067 conn(0x7fd6f00d3e30 msgr2=0x7fd6f0153c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 shutdown_connections 2024-04-25T01:08:37.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.445+0000 7fd6f7869640 1 -- 172.21.15.33:0/3496722067 wait complete. 2024-04-25T01:08:37.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:08:37.456 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:08:37.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f28f38640 1 Processor -- start 2024-04-25T01:08:37.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f28f38640 1 -- start start 2024-04-25T01:08:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f28f38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 0x7f3f240cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f28f38640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f240b3560 con 0x7f3f2415a3a0 2024-04-25T01:08:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f28f38640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f240b3840 con 0x7f3f2415eb90 2024-04-25T01:08:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f28f38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f240b4450 con 0x7f3f2415e650 2024-04-25T01:08:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3f2415a3a0 0x7f3f240b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59430/0 (socket says 172.21.15.33:59430) 2024-04-25T01:08:37.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 -- 172.21.15.33:0/3951440292 learned_addr learned my addr 172.21.15.33:0/3951440292 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f22d9d640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 0x7f3f240cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2421965019 0 0) 0x7f3f240b3560 con 0x7f3f2415a3a0 2024-04-25T01:08:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f0c005740 con 0x7f3f2415a3a0 2024-04-25T01:08:37.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201184794 0 0) 0x7f3f240b3840 con 0x7f3f2415eb90 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f240b3560 con 0x7f3f2415eb90 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 551288362 0 0) 0x7f3f0c005740 con 0x7f3f2415a3a0 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 msgr2=0x7f3f240cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 0x7f3f240cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 >> v1:172.21.15.138:6789/0 conn(0x7f3f2415eb90 legacy=0x7f3f2415df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f240b3b20 con 0x7f3f2415a3a0 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f3f18001680 con 0x7f3f2415a3a0 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 >> v1:172.21.15.33:6789/0 conn(0x7f3f2415a3a0 legacy=0x7f3f240b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f0c006190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 0x7f3f0c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f0c006f60 0x7f3f0c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f240b4450 con 0x7f3f0c005d90 2024-04-25T01:08:37.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f0c00dc90 con 0x7f3f0c006f60 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f0c00de90 con 0x7f3f2415e650 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f0c006190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f22d9d640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f0c006f60 0x7f3f0c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f21d9b640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 0x7f3f0c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 -- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 msgr2=0x7f3f0c006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f2415e650 0x7f3f0c006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 -- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f0c006f60 msgr2=0x7f3f0c0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f0c006f60 0x7f3f0c0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.589+0000 7f3f2259c640 1 -- 172.21.15.33:0/3951440292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f0c005740 con 0x7f3f0c005d90 2024-04-25T01:08:37.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f2259c640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f0c006190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c190 tx=0x7f3f1801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a006273567dfc9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18007cd0 con 0x7f3f0c005d90 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f180a9020 con 0x7f3f0c005d90 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f2159a640 1 -- 172.21.15.33:0/3951440292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18005ae0 con 0x7f3f0c005d90 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- 172.21.15.33:0/3951440292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 msgr2=0x7f3f0c006190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 --2- 172.21.15.33:0/3951440292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f0c006190 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f1800c190 tx=0x7f3f1801cbf0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f22d9d640 1 -- 172.21.15.33:0/3951440292 reap_dead start 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- 172.21.15.33:0/3951440292 shutdown_connections 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- 172.21.15.33:0/3951440292 >> 172.21.15.33:0/3951440292 conn(0x7f3f240b1810 msgr2=0x7f3f240b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- 172.21.15.33:0/3951440292 shutdown_connections 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- 172.21.15.33:0/3951440292 wait complete. 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 Processor -- start 2024-04-25T01:08:37.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- start start 2024-04-25T01:08:37.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f240b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f241522c0 0x7f3f241f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f24152c60 0x7f3f241530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2406d420 con 0x7f3f0c005d90 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f240cf790 con 0x7f3f24152c60 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f28f38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f2406d5a0 con 0x7f3f241522c0 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.593+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f240b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f240b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34586/0 (socket says 172.21.15.33:34586) 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 -- 172.21.15.33:0/1416131541 learned_addr learned my addr 172.21.15.33:0/1416131541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f21d9b640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f241522c0 0x7f3f241f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f22d9d640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f24152c60 0x7f3f241530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 -- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f241522c0 msgr2=0x7f3f241f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f241522c0 0x7f3f241f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 -- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f24152c60 msgr2=0x7f3f241530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f24152c60 0x7f3f241530f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 -- 172.21.15.33:0/1416131541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f241f7590 con 0x7f3f0c005d90 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f2259c640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f240b0df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f18007040 tx=0x7f3f0c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f12ffd640 1 -- 172.21.15.33:0/1416131541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18007cd0 con 0x7f3f0c005d90 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f12ffd640 1 -- 172.21.15.33:0/1416131541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f180047f0 con 0x7f3f0c005d90 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f241f7830 con 0x7f3f0c005d90 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f12ffd640 1 -- 172.21.15.33:0/1416131541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18001c10 con 0x7f3f0c005d90 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f241f7df0 con 0x7f3f0c005d90 2024-04-25T01:08:37.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f24069040 con 0x7f3f0c005d90 2024-04-25T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f12ffd640 1 -- 172.21.15.33:0/1416131541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3f18004bc0 con 0x7f3f0c005d90 2024-04-25T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f12ffd640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f040678c0 0x7f3f04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f12ffd640 1 -- 172.21.15.33:0/1416131541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3f1811fda0 con 0x7f3f0c005d90 2024-04-25T01:08:37.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f21d9b640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f040678c0 0x7f3f04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.597+0000 7f3f21d9b640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f040678c0 0x7f3f04069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f14005450 tx=0x7f3f14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.601+0000 7f3f12ffd640 1 -- 172.21.15.33:0/1416131541 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3f18016930 con 0x7f3f0c005d90 2024-04-25T01:08:37.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f241638b0 con 0x7f3f0c005d90 2024-04-25T01:08:37.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f12ffd640 1 -- 172.21.15.33:0/1416131541 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f3f180f0630 con 0x7f3f0c005d90 2024-04-25T01:08:37.753 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:08:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f040678c0 msgr2=0x7f3f04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f040678c0 0x7f3f04069d80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f14005450 tx=0x7f3f14009040 comp rx=0 tx=0).stop 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 msgr2=0x7f3f240b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f240b0df0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3f18007040 tx=0x7f3f0c007a50 comp rx=0 tx=0).stop 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 shutdown_connections 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f040678c0 0x7f3f04069d80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f24152c60 0x7f3f241530f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f241522c0 0x7f3f241f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 --2- 172.21.15.33:0/1416131541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f0c005d90 0x7f3f240b0df0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 >> 172.21.15.33:0/1416131541 conn(0x7f3f240b1810 msgr2=0x7f3f2414e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 shutdown_connections 2024-04-25T01:08:37.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.761+0000 7f3f28f38640 1 -- 172.21.15.33:0/1416131541 wait complete. 2024-04-25T01:08:37.770 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:08:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca70c9e640 1 Processor -- start 2024-04-25T01:08:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca70c9e640 1 -- start start 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca70c9e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca6c15cc50 0x7fca6c06b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca70c9e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca6c15d0b0 0x7fca6c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca70c9e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca6c0d07e0 con 0x7fca6c0af240 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca70c9e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fca6c15c470 con 0x7fca6c15cc50 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca70c9e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fca6c0afeb0 con 0x7fca6c15d0b0 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca6a59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca6c0af240 0x7fca6c0d00d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59440/0 (socket says 172.21.15.33:59440) 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca6a59c640 1 -- 172.21.15.33:0/2362691913 learned_addr learned my addr 172.21.15.33:0/2362691913 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca69d9b640 1 --2- 172.21.15.33:0/2362691913 >> v2:172.21.15.33:3300/0 conn(0x7fca6c15cc50 0x7fca6c06b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca6ad9d640 1 --2- 172.21.15.33:0/2362691913 >> v2:172.21.15.138:3300/0 conn(0x7fca6c15d0b0 0x7fca6c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca6959a640 1 -- 172.21.15.33:0/2362691913 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965900201 0 0) 0x7fca6c0d07e0 con 0x7fca6c0af240 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca6959a640 1 -- 172.21.15.33:0/2362691913 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca54003530 con 0x7fca6c0af240 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca69d9b640 1 -- 172.21.15.33:0/2362691913 >> v2:172.21.15.138:3300/0 conn(0x7fca6c15d0b0 msgr2=0x7fca6c06b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:37.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca69d9b640 1 --2- 172.21.15.33:0/2362691913 >> v2:172.21.15.138:3300/0 conn(0x7fca6c15d0b0 0x7fca6c06b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca69d9b640 1 -- 172.21.15.33:0/2362691913 >> v1:172.21.15.33:6789/0 conn(0x7fca6c0af240 legacy=0x7fca6c0d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca6959a640 1 -- 172.21.15.33:0/2362691913 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1949674428 0 0) 0x7fca54003530 con 0x7fca6c0af240 2024-04-25T01:08:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.905+0000 7fca69d9b640 1 -- 172.21.15.33:0/2362691913 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c0b0190 con 0x7fca6c15cc50 2024-04-25T01:08:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca69d9b640 1 --2- 172.21.15.33:0/2362691913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c15cc50 0x7fca6c06b210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca6c06f310 tx=0x7fca6001c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b6038002693fe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca6959a640 1 -- 172.21.15.33:0/2362691913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca600b0070 con 0x7fca6c15cc50 2024-04-25T01:08:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca6959a640 1 -- 172.21.15.33:0/2362691913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca600ab400 con 0x7fca6c15cc50 2024-04-25T01:08:37.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca6959a640 1 -- 172.21.15.33:0/2362691913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64001a00 con 0x7fca6c15cc50 2024-04-25T01:08:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 -- 172.21.15.33:0/2362691913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c15cc50 msgr2=0x7fca6c06b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 --2- 172.21.15.33:0/2362691913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c15cc50 0x7fca6c06b210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca6c06f310 tx=0x7fca6001c2e0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 -- 172.21.15.33:0/2362691913 shutdown_connections 2024-04-25T01:08:37.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 --2- 172.21.15.33:0/2362691913 >> v2:172.21.15.138:3300/0 conn(0x7fca6c15d0b0 0x7fca6c06b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 --2- 172.21.15.33:0/2362691913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c15cc50 0x7fca6c06b210 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 -- 172.21.15.33:0/2362691913 >> 172.21.15.33:0/2362691913 conn(0x7fca6c0677a0 msgr2=0x7fca6c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 -- 172.21.15.33:0/2362691913 shutdown_connections 2024-04-25T01:08:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 -- 172.21.15.33:0/2362691913 wait complete. 2024-04-25T01:08:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 Processor -- start 2024-04-25T01:08:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.909+0000 7fca70c9e640 1 -- start start 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c0af240 0x7fca6c14b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c15cc50 0x7fca6c0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15d0b0 0x7fca6c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0af620 con 0x7fca6c0af240 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c0afb40 con 0x7fca6c15cc50 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca6c0b57c0 con 0x7fca6c15d0b0 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c0af240 0x7fca6c14b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c0af240 0x7fca6c14b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34602/0 (socket says 172.21.15.33:34602) 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 -- 172.21.15.33:0/4038830835 learned_addr learned my addr 172.21.15.33:0/4038830835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 -- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15d0b0 msgr2=0x7fca6c156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15d0b0 0x7fca6c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 -- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c15cc50 msgr2=0x7fca6c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c15cc50 0x7fca6c0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 -- 172.21.15.33:0/4038830835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c152d20 con 0x7fca6c0af240 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca6a59c640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c0af240 0x7fca6c14b510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca64001f60 tx=0x7fca6400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca5affd640 1 -- 172.21.15.33:0/4038830835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca64017070 con 0x7fca6c0af240 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca5affd640 1 -- 172.21.15.33:0/4038830835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca64012400 con 0x7fca6c0af240 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c0b32f0 con 0x7fca6c0af240 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca5affd640 1 -- 172.21.15.33:0/4038830835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640127d0 con 0x7fca6c0af240 2024-04-25T01:08:37.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c0b52a0 con 0x7fca6c0af240 2024-04-25T01:08:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c069040 con 0x7fca6c0af240 2024-04-25T01:08:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca5affd640 1 -- 172.21.15.33:0/4038830835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fca6401a070 con 0x7fca6c0af240 2024-04-25T01:08:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca5affd640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca4c0678c0 0x7fca4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca5affd640 1 -- 172.21.15.33:0/4038830835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fca64085ee0 con 0x7fca6c0af240 2024-04-25T01:08:37.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.913+0000 7fca69d9b640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca4c0678c0 0x7fca4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:37.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.917+0000 7fca69d9b640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca4c0678c0 0x7fca4c069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca600ad000 tx=0x7fca60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:37.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:37.917+0000 7fca5affd640 1 -- 172.21.15.33:0/4038830835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fca64056770 con 0x7fca6c0af240 2024-04-25T01:08:38.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.077+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fca6c0b5470 con 0x7fca6c0af240 2024-04-25T01:08:38.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca5affd640 1 -- 172.21.15.33:0/4038830835 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fca64056070 con 0x7fca6c0af240 2024-04-25T01:08:38.072 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca4c0678c0 msgr2=0x7fca4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca4c0678c0 0x7fca4c069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca600ad000 tx=0x7fca60008040 comp rx=0 tx=0).stop 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c0af240 msgr2=0x7fca6c14b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c0af240 0x7fca6c14b510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fca64001f60 tx=0x7fca6400a430 comp rx=0 tx=0).stop 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 shutdown_connections 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca4c0678c0 0x7fca4c069d80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca6c15d0b0 0x7fca6c156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca6c15cc50 0x7fca6c0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 --2- 172.21.15.33:0/4038830835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca6c0af240 0x7fca6c14b510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 >> 172.21.15.33:0/4038830835 conn(0x7fca6c0677a0 msgr2=0x7fca6c0ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 shutdown_connections 2024-04-25T01:08:38.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.081+0000 7fca70c9e640 1 -- 172.21.15.33:0/4038830835 wait complete. 2024-04-25T01:08:38.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:08:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7c090640 1 Processor -- start 2024-04-25T01:08:38.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7c090640 1 -- start start 2024-04-25T01:08:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7c090640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fad7406ecc0 0x7fad740b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7c090640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 0x7fad740b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7c090640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad740b4a20 con 0x7fad7415ae20 2024-04-25T01:08:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7c090640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fad740b2bb0 con 0x7fad7406ecc0 2024-04-25T01:08:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7c090640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad740b2e90 con 0x7fad74072410 2024-04-25T01:08:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fad7415ae20 0x7fad7406e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59454/0 (socket says 172.21.15.33:59454) 2024-04-25T01:08:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 -- 172.21.15.33:0/3330276310 learned_addr learned my addr 172.21.15.33:0/3330276310 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7a62d640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 0x7fad740b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7962b640 1 --2- 172.21.15.33:0/3330276310 >> v2:172.21.15.138:3300/0 conn(0x7fad7406ecc0 0x7fad740b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 554595451 0 0) 0x7fad740b4a20 con 0x7fad7415ae20 2024-04-25T01:08:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad60003530 con 0x7fad7415ae20 2024-04-25T01:08:38.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 266778228 0 0) 0x7fad60003530 con 0x7fad7415ae20 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 msgr2=0x7fad740b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 0x7fad740b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 >> v2:172.21.15.138:3300/0 conn(0x7fad7406ecc0 msgr2=0x7fad740b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 --2- 172.21.15.33:0/3330276310 >> v2:172.21.15.138:3300/0 conn(0x7fad7406ecc0 0x7fad740b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740b3170 con 0x7fad7415ae20 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fad70001680 con 0x7fad7415ae20 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 >> v1:172.21.15.33:6789/0 conn(0x7fad7415ae20 legacy=0x7fad7406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad60004810 0x7fad60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 0x7fad600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b2bb0 con 0x7fad600039c0 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b2e90 con 0x7fad60004810 2024-04-25T01:08:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad6000b9d0 con 0x7fad74072410 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7962b640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad60004810 0x7fad60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7a62d640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 0x7fad600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 -- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 msgr2=0x7fad600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 0x7fad600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 -- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad60004810 msgr2=0x7fad60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad60004810 0x7fad60004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad79e2c640 1 -- 172.21.15.33:0/3330276310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad60003530 con 0x7fad600039c0 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.217+0000 7fad7a62d640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74072410 0x7fad600051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad79e2c640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad60003de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c190 tx=0x7fad7001cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903f9d6c3ff4665f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70007c90 con 0x7fad600039c0 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad700a9020 con 0x7fad600039c0 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad78e2a640 1 -- 172.21.15.33:0/3330276310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70005b00 con 0x7fad600039c0 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 -- 172.21.15.33:0/3330276310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 msgr2=0x7fad60003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 --2- 172.21.15.33:0/3330276310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad60003de0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fad7000c190 tx=0x7fad7001cba0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7a62d640 1 -- 172.21.15.33:0/3330276310 reap_dead start 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 -- 172.21.15.33:0/3330276310 shutdown_connections 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 -- 172.21.15.33:0/3330276310 >> 172.21.15.33:0/3330276310 conn(0x7fad740b1810 msgr2=0x7fad740b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 -- 172.21.15.33:0/3330276310 shutdown_connections 2024-04-25T01:08:38.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 -- 172.21.15.33:0/3330276310 wait complete. 2024-04-25T01:08:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 Processor -- start 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.221+0000 7fad7c090640 1 -- start start 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad74163570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad74160380 0x7fad74150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74159cc0 0x7fad74156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fad740cd840 con 0x7fad600039c0 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fad740cd6c0 con 0x7fad74160380 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fad740cd9c0 con 0x7fad74159cc0 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad74163570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad74163570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34622/0 (socket says 172.21.15.33:34622) 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 -- 172.21.15.33:0/1893987222 learned_addr learned my addr 172.21.15.33:0/1893987222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7962b640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad74160380 0x7fad74150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 -- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74159cc0 msgr2=0x7fad74156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74159cc0 0x7fad74156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 -- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad74160380 msgr2=0x7fad74150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad74160380 0x7fad74150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 -- 172.21.15.33:0/1893987222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad741517b0 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7962b640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad74160380 0x7fad74150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad79e2c640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad74163570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad700ab040 tx=0x7fad60005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad6a7fc640 1 -- 172.21.15.33:0/1893987222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad700a5a00 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad6a7fc640 1 -- 172.21.15.33:0/1893987222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad700a5ba0 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad74160a10 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad6a7fc640 1 -- 172.21.15.33:0/1893987222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad700047f0 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad741502c0 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad74069040 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad6a7fc640 1 -- 172.21.15.33:0/1893987222 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fad70004990 con 0x7fad600039c0 2024-04-25T01:08:38.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad6a7fc640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fad580678f0 0x7fad58069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad6a7fc640 1 -- 172.21.15.33:0/1893987222 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fad7011f5e0 con 0x7fad600039c0 2024-04-25T01:08:38.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.225+0000 7fad7962b640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fad580678f0 0x7fad58069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.229+0000 7fad7962b640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fad580678f0 0x7fad58069db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad74163bb0 tx=0x7fad64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.229+0000 7fad6a7fc640 1 -- 172.21.15.33:0/1893987222 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fad70016910 con 0x7fad600039c0 2024-04-25T01:08:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad74160760 con 0x7fad600039c0 2024-04-25T01:08:38.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad6a7fc640 1 -- 172.21.15.33:0/1893987222 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fad700efcb0 con 0x7fad600039c0 2024-04-25T01:08:38.378 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fad580678f0 msgr2=0x7fad58069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fad580678f0 0x7fad58069db0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad74163bb0 tx=0x7fad64008040 comp rx=0 tx=0).stop 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 msgr2=0x7fad74163570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad74163570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad700ab040 tx=0x7fad60005840 comp rx=0 tx=0).stop 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 shutdown_connections 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fad580678f0 0x7fad58069db0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fad74159cc0 0x7fad74156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad74160380 0x7fad74150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 --2- 172.21.15.33:0/1893987222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad600039c0 0x7fad74163570 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 >> 172.21.15.33:0/1893987222 conn(0x7fad740b1810 msgr2=0x7fad7415d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:38.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 shutdown_connections 2024-04-25T01:08:38.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.385+0000 7fad7c090640 1 -- 172.21.15.33:0/1893987222 wait complete. 2024-04-25T01:08:38.395 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:08:38.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca3646c640 1 Processor -- start 2024-04-25T01:08:38.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca3646c640 1 -- start start 2024-04-25T01:08:38.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca3646c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca3646c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca3015f960 0x7fca3015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca3646c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca30160770 con 0x7fca301602e0 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca3646c640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fca3006f0a0 con 0x7fca3015f960 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca3646c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca30145a40 con 0x7fca30145660 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2f7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca3015f960 0x7fca3015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca34a09640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca301602e0 0x7fca3006d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59462/0 (socket says 172.21.15.33:59462) 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2f7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca3015f960 0x7fca3015fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34624/0 (socket says 172.21.15.33:34624) 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca34a09640 1 -- 172.21.15.33:0/2434464092 learned_addr learned my addr 172.21.15.33:0/2434464092 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2ffff640 1 --2- 172.21.15.33:0/2434464092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2f7fe640 1 -- 172.21.15.33:0/2434464092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 msgr2=0x7fca3006eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2f7fe640 1 --2- 172.21.15.33:0/2434464092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3006eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2f7fe640 1 -- 172.21.15.33:0/2434464092 >> v1:172.21.15.33:6789/0 conn(0x7fca301602e0 legacy=0x7fca3006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2f7fe640 1 -- 172.21.15.33:0/2434464092 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca300cee10 con 0x7fca3015f960 2024-04-25T01:08:38.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.517+0000 7fca2effd640 1 -- 172.21.15.33:0/2434464092 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199033672 0 0) 0x7fca30160770 con 0x7fca301602e0 2024-04-25T01:08:38.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca2f7fe640 1 --2- 172.21.15.33:0/2434464092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3015f960 0x7fca3015fda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fca180097c0 tx=0x7fca1801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec39fb4274596d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca2effd640 1 -- 172.21.15.33:0/2434464092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca180b2070 con 0x7fca3015f960 2024-04-25T01:08:38.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca2effd640 1 -- 172.21.15.33:0/2434464092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca180ad640 con 0x7fca3015f960 2024-04-25T01:08:38.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca2effd640 1 -- 172.21.15.33:0/2434464092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca180ada10 con 0x7fca3015f960 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 -- 172.21.15.33:0/2434464092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3015f960 msgr2=0x7fca3015fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 --2- 172.21.15.33:0/2434464092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3015f960 0x7fca3015fda0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fca180097c0 tx=0x7fca1801c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 -- 172.21.15.33:0/2434464092 shutdown_connections 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 --2- 172.21.15.33:0/2434464092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca3015f960 0x7fca3015fda0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 --2- 172.21.15.33:0/2434464092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3006eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 -- 172.21.15.33:0/2434464092 >> 172.21.15.33:0/2434464092 conn(0x7fca300b1810 msgr2=0x7fca300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 -- 172.21.15.33:0/2434464092 shutdown_connections 2024-04-25T01:08:38.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 -- 172.21.15.33:0/2434464092 wait complete. 2024-04-25T01:08:38.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 Processor -- start 2024-04-25T01:08:38.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.521+0000 7fca3646c640 1 -- start start 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3015f960 0x7fca30138020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca301602e0 0x7fca301418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca30168e20 con 0x7fca301602e0 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca30168ca0 con 0x7fca3015f960 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca30168fa0 con 0x7fca30145660 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca301602e0 0x7fca301418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca301602e0 0x7fca301418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34634/0 (socket says 172.21.15.33:34634) 2024-04-25T01:08:38.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 -- 172.21.15.33:0/2657431166 learned_addr learned my addr 172.21.15.33:0/2657431166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2f7fe640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3015f960 0x7fca30138020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2ffff640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 -- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 msgr2=0x7fca3015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3015f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 -- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3015f960 msgr2=0x7fca30138020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3015f960 0x7fca30138020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 -- 172.21.15.33:0/2657431166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca30141e20 con 0x7fca301602e0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca34a09640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca301602e0 0x7fca301418e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c470 tx=0x7fca2800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2cff9640 1 -- 172.21.15.33:0/2657431166 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28018070 con 0x7fca301602e0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2cff9640 1 -- 172.21.15.33:0/2657431166 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca280074e0 con 0x7fca301602e0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3015c580 con 0x7fca301602e0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2cff9640 1 -- 172.21.15.33:0/2657431166 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28004a30 con 0x7fca301602e0 2024-04-25T01:08:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3015e9a0 con 0x7fca301602e0 2024-04-25T01:08:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca30069040 con 0x7fca301602e0 2024-04-25T01:08:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2cff9640 1 -- 172.21.15.33:0/2657431166 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fca28004bd0 con 0x7fca301602e0 2024-04-25T01:08:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2cff9640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678f0 0x7fca10069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2cff9640 1 -- 172.21.15.33:0/2657431166 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fca280852a0 con 0x7fca301602e0 2024-04-25T01:08:38.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.525+0000 7fca2ffff640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678f0 0x7fca10069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.529+0000 7fca2ffff640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678f0 0x7fca10069db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca200073f0 tx=0x7fca2000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.529+0000 7fca2cff9640 1 -- 172.21.15.33:0/2657431166 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fca28055a20 con 0x7fca301602e0 2024-04-25T01:08:38.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fca30072ac0 con 0x7fca301602e0 2024-04-25T01:08:38.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca2cff9640 1 -- 172.21.15.33:0/2657431166 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fca28055320 con 0x7fca301602e0 2024-04-25T01:08:38.677 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678f0 msgr2=0x7fca10069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678f0 0x7fca10069db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fca200073f0 tx=0x7fca2000b040 comp rx=0 tx=0).stop 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca301602e0 msgr2=0x7fca301418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca301602e0 0x7fca301418e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c470 tx=0x7fca2800c930 comp rx=0 tx=0).stop 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 shutdown_connections 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678f0 0x7fca10069db0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca301602e0 0x7fca301418e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3015f960 0x7fca30138020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 --2- 172.21.15.33:0/2657431166 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca30145660 0x7fca3015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 >> 172.21.15.33:0/2657431166 conn(0x7fca300b1810 msgr2=0x7fca3006e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 shutdown_connections 2024-04-25T01:08:38.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.685+0000 7fca3646c640 1 -- 172.21.15.33:0/2657431166 wait complete. 2024-04-25T01:08:38.695 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:08:38.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc20d912640 1 Processor -- start 2024-04-25T01:08:38.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc20d912640 1 -- start start 2024-04-25T01:08:38.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc20d912640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc20815e470 0x7fc2080ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc20d912640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080d48d0 0x7fc2080cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc20d912640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2080d10b0 con 0x7fc2080d03e0 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc20d912640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc2080d1de0 con 0x7fc20815e470 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc20d912640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc2080d1390 con 0x7fc2080d48d0 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc1fffff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc20815e470 0x7fc2080ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc1fffff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc20815e470 0x7fc2080ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49742/0 (socket says 172.21.15.33:49742) 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc1fffff640 1 -- 172.21.15.33:0/4024732308 learned_addr learned my addr 172.21.15.33:0/4024732308 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc206ffd640 1 --1- 172.21.15.33:0/4024732308 >> v1:172.21.15.33:6789/0 conn(0x7fc2080d03e0 0x7fc2080d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33784/0 (socket says 172.21.15.33:33784) 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc2077fe640 1 --2- 172.21.15.33:0/4024732308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080d48d0 0x7fc2080cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc1fffff640 1 -- 172.21.15.33:0/4024732308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080d48d0 msgr2=0x7fc2080cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc1fffff640 1 --2- 172.21.15.33:0/4024732308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080d48d0 0x7fc2080cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc1fffff640 1 -- 172.21.15.33:0/4024732308 >> v1:172.21.15.33:6789/0 conn(0x7fc2080d03e0 legacy=0x7fc2080d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc1fffff640 1 -- 172.21.15.33:0/4024732308 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080d1670 con 0x7fc20815e470 2024-04-25T01:08:38.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc2067fc640 1 -- 172.21.15.33:0/4024732308 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261323766 0 0) 0x7fc2080d10b0 con 0x7fc2080d03e0 2024-04-25T01:08:38.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.813+0000 7fc2077fe640 1 --2- 172.21.15.33:0/4024732308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080d48d0 0x7fc2080cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:38.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc1fffff640 1 --2- 172.21.15.33:0/4024732308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20815e470 0x7fc2080ce9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009870 tx=0x7fc1f001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8779cc067df01e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc2067fc640 1 -- 172.21.15.33:0/4024732308 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00b1070 con 0x7fc20815e470 2024-04-25T01:08:38.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc2067fc640 1 -- 172.21.15.33:0/4024732308 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f00a8050 con 0x7fc20815e470 2024-04-25T01:08:38.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc2067fc640 1 -- 172.21.15.33:0/4024732308 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00ac860 con 0x7fc20815e470 2024-04-25T01:08:38.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 -- 172.21.15.33:0/4024732308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20815e470 msgr2=0x7fc2080ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 --2- 172.21.15.33:0/4024732308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20815e470 0x7fc2080ce9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0009870 tx=0x7fc1f001c4d0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 -- 172.21.15.33:0/4024732308 shutdown_connections 2024-04-25T01:08:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 --2- 172.21.15.33:0/4024732308 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2080d48d0 0x7fc2080cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 --2- 172.21.15.33:0/4024732308 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20815e470 0x7fc2080ce9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 -- 172.21.15.33:0/4024732308 >> 172.21.15.33:0/4024732308 conn(0x7fc2080d3db0 msgr2=0x7fc2080d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 -- 172.21.15.33:0/4024732308 shutdown_connections 2024-04-25T01:08:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 -- 172.21.15.33:0/4024732308 wait complete. 2024-04-25T01:08:38.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 Processor -- start 2024-04-25T01:08:38.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.817+0000 7fc20d912640 1 -- start start 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d03e0 0x7fc2080cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080d48d0 0x7fc208156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc20815e470 0x7fc2081fa260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc20816adc0 con 0x7fc2080d48d0 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2080cfe60 con 0x7fc2080d03e0 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc20816af40 con 0x7fc20815e470 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080d48d0 0x7fc208156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080d48d0 0x7fc208156470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49752/0 (socket says 172.21.15.33:49752) 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 -- 172.21.15.33:0/3712980761 learned_addr learned my addr 172.21.15.33:0/3712980761 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc206ffd640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d03e0 0x7fc2080cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 -- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc20815e470 msgr2=0x7fc2081fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc20815e470 0x7fc2081fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 -- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d03e0 msgr2=0x7fc2080cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d03e0 0x7fc2080cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 -- 172.21.15.33:0/3712980761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080b9400 con 0x7fc2080d48d0 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc206ffd640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d03e0 0x7fc2080cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc1fffff640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080d48d0 0x7fc208156470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc1f001a100 tx=0x7fc1f001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc204ff9640 1 -- 172.21.15.33:0/3712980761 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00bb020 con 0x7fc2080d48d0 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc204ff9640 1 -- 172.21.15.33:0/3712980761 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f00b7640 con 0x7fc2080d48d0 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2080b96e0 con 0x7fc2080d48d0 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc204ff9640 1 -- 172.21.15.33:0/3712980761 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f00b1040 con 0x7fc2080d48d0 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2080cb3c0 con 0x7fc2080d48d0 2024-04-25T01:08:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc208069040 con 0x7fc2080d48d0 2024-04-25T01:08:38.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc204ff9640 1 -- 172.21.15.33:0/3712980761 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc1f00be020 con 0x7fc2080d48d0 2024-04-25T01:08:38.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc204ff9640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc1e80678c0 0x7fc1e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:38.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc204ff9640 1 -- 172.21.15.33:0/3712980761 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc1f0120510 con 0x7fc2080d48d0 2024-04-25T01:08:38.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.821+0000 7fc206ffd640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc1e80678c0 0x7fc1e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:38.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.825+0000 7fc206ffd640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc1e80678c0 0x7fc1e8069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8000f70 tx=0x7fc1f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:38.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:38.825+0000 7fc204ff9640 1 -- 172.21.15.33:0/3712980761 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc1f00c5030 con 0x7fc2080d48d0 2024-04-25T01:08:38.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.001+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2080d0960 con 0x7fc2080d48d0 2024-04-25T01:08:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc204ff9640 1 -- 172.21.15.33:0/3712980761 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fc1f0016150 con 0x7fc2080d48d0 2024-04-25T01:08:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:08:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc1e80678c0 msgr2=0x7fc1e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc1e80678c0 0x7fc1e8069d80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8000f70 tx=0x7fc1f8009040 comp rx=0 tx=0).stop 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080d48d0 msgr2=0x7fc208156470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080d48d0 0x7fc208156470 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc1f001a100 tx=0x7fc1f001ad10 comp rx=0 tx=0).stop 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 shutdown_connections 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc1e80678c0 0x7fc1e8069d80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc20815e470 0x7fc2081fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2080d48d0 0x7fc208156470 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 --2- 172.21.15.33:0/3712980761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2080d03e0 0x7fc2080cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 >> 172.21.15.33:0/3712980761 conn(0x7fc2080d3db0 msgr2=0x7fc2080c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 shutdown_connections 2024-04-25T01:08:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.005+0000 7fc20d912640 1 -- 172.21.15.33:0/3712980761 wait complete. 2024-04-25T01:08:39.014 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-04-25T01:08:39.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:39.015 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T01:08:39.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi033.stderr:ceph-fuse[33506]: fuse finished with error 0 and tester_r 0 2024-04-25T01:08:39.035 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:08:39.108 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:08:39.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:39.109 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T01:08:39.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ceddf640 1 Processor -- start 2024-04-25T01:08:39.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ceddf640 1 -- start start 2024-04-25T01:08:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ceddf640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f18c806b920 0x7f18c815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ceddf640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c80d0a00 con 0x7f18c815c2d0 2024-04-25T01:08:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ceddf640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18c815cdc0 con 0x7f18c80d0620 2024-04-25T01:08:39.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ceddf640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f18c806e3a0 con 0x7f18c806b920 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f18c815c2d0 0x7f18c806b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33800/0 (socket says 172.21.15.33:33800) 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 -- 172.21.15.33:0/2703764586 learned_addr learned my addr 172.21.15.33:0/2703764586 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18cd37c640 1 --2- 172.21.15.33:0/2703764586 >> v2:172.21.15.138:3300/0 conn(0x7f18c806b920 0x7f18c815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323916841 0 0) 0x7f18c80d0a00 con 0x7f18c815c2d0 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18b0003530 con 0x7f18c815c2d0 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2322326579 0 0) 0x7f18b0003530 con 0x7f18c815c2d0 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 >> v2:172.21.15.138:3300/0 conn(0x7f18c806b920 msgr2=0x7f18c815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 --2- 172.21.15.33:0/2703764586 >> v2:172.21.15.138:3300/0 conn(0x7f18c806b920 0x7f18c815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 >> v1:172.21.15.138:6789/0 conn(0x7f18c80d0620 legacy=0x7f18c806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c815d0a0 con 0x7f18c815c2d0 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f18bc001a00 con 0x7f18c815c2d0 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 >> v1:172.21.15.33:6789/0 conn(0x7f18c815c2d0 legacy=0x7f18c806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b00039c0 0x7f18b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b00051d0 0x7f18b000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18c806e3a0 con 0x7f18b0004810 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18b000c070 con 0x7f18b00051d0 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18b000c270 con 0x7f18b00039c0 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18cd37c640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b00051d0 0x7f18b000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ccb7b640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b00039c0 0x7f18b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 -- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b00039c0 msgr2=0x7f18b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b00039c0 0x7f18b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 -- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b00051d0 msgr2=0x7f18b000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18b00051d0 0x7f18b000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18c7fff640 1 -- 172.21.15.33:0/2703764586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b0003530 con 0x7f18b0004810 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18cd37c640 1 -- 172.21.15.33:0/2703764586 reap_dead start 2024-04-25T01:08:39.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.301+0000 7f18ccb7b640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18b00039c0 0x7f18b0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18c7fff640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00a860 tx=0x7f18bc01faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5e908f43530aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc00a480 con 0x7f18b0004810 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18bc0aa6d0 con 0x7f18b0004810 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18c77fe640 1 -- 172.21.15.33:0/2703764586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18bc0aab40 con 0x7f18b0004810 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 -- 172.21.15.33:0/2703764586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 msgr2=0x7f18b0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f18bc00a860 tx=0x7f18bc01faa0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 -- 172.21.15.33:0/2703764586 shutdown_connections 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 --2- 172.21.15.33:0/2703764586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18b0004c90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 -- 172.21.15.33:0/2703764586 >> 172.21.15.33:0/2703764586 conn(0x7f18c80b29c0 msgr2=0x7f18c80b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 -- 172.21.15.33:0/2703764586 shutdown_connections 2024-04-25T01:08:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 -- 172.21.15.33:0/2703764586 wait complete. 2024-04-25T01:08:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 Processor -- start 2024-04-25T01:08:39.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 -- start start 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.305+0000 7f18ceddf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18c8156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18c81f1b00 0x7f18c81f1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18c8140fc0 0x7f18c8141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18c80adf30 con 0x7f18b0004810 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18c806e8d0 con 0x7f18c81f1b00 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f18c815c760 con 0x7f18c8140fc0 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18c8156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18c8156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49762/0 (socket says 172.21.15.33:49762) 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 -- 172.21.15.33:0/1787548392 learned_addr learned my addr 172.21.15.33:0/1787548392 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:39.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18cd37c640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18c8140fc0 0x7f18c8141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c7fff640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18c81f1b00 0x7f18c81f1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 -- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18c8140fc0 msgr2=0x7f18c8141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18c8140fc0 0x7f18c8141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 -- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18c81f1b00 msgr2=0x7f18c81f1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18c81f1b00 0x7f18c81f1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 -- 172.21.15.33:0/1787548392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c81500f0 con 0x7f18b0004810 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18cd37c640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18c8140fc0 0x7f18c8141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c7fff640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18c81f1b00 0x7f18c81f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ccb7b640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18c8156990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d4a0 tx=0x7f18b800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c57fa640 1 -- 172.21.15.33:0/1787548392 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8005280 con 0x7f18b0004810 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c57fa640 1 -- 172.21.15.33:0/1787548392 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18b8005440 con 0x7f18b0004810 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c81503d0 con 0x7f18b0004810 2024-04-25T01:08:39.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c57fa640 1 -- 172.21.15.33:0/1787548392 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8007260 con 0x7f18b0004810 2024-04-25T01:08:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c8150990 con 0x7f18b0004810 2024-04-25T01:08:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18c8069040 con 0x7f18b0004810 2024-04-25T01:08:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c57fa640 1 -- 172.21.15.33:0/1787548392 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f18b8019020 con 0x7f18b0004810 2024-04-25T01:08:39.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c57fa640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18b40678f0 0x7f18b4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c57fa640 1 -- 172.21.15.33:0/1787548392 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f18b8085740 con 0x7f18b0004810 2024-04-25T01:08:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.309+0000 7f18c7fff640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18b40678f0 0x7f18b4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.313+0000 7f18c7fff640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18b40678f0 0x7f18b4069db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0b1000 tx=0x7f18bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.313+0000 7f18c57fa640 1 -- 172.21.15.33:0/1787548392 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f18b8055ea0 con 0x7f18b0004810 2024-04-25T01:08:39.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.469+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18c8073df0 con 0x7f18b0004810 2024-04-25T01:08:39.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.473+0000 7f18c57fa640 1 -- 172.21.15.33:0/1787548392 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f18b80557a0 con 0x7f18b0004810 2024-04-25T01:08:39.467 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:39.467 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18b40678f0 msgr2=0x7f18b4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18b40678f0 0x7f18b4069db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f18bc0b1000 tx=0x7f18bc009040 comp rx=0 tx=0).stop 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 msgr2=0x7f18c8156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18c8156990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18b800d4a0 tx=0x7f18b800d960 comp rx=0 tx=0).stop 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 shutdown_connections 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18b40678f0 0x7f18b4069db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f18c8140fc0 0x7f18c8141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18c81f1b00 0x7f18c81f1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 --2- 172.21.15.33:0/1787548392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18b0004810 0x7f18c8156990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 >> 172.21.15.33:0/1787548392 conn(0x7f18c80b29c0 msgr2=0x7f18c809af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 shutdown_connections 2024-04-25T01:08:39.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.477+0000 7f18ceddf640 1 -- 172.21.15.33:0/1787548392 wait complete. 2024-04-25T01:08:39.486 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.613+0000 7f566850b640 1 Processor -- start 2024-04-25T01:08:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.613+0000 7f566850b640 1 -- start start 2024-04-25T01:08:39.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f566850b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56600b1010 0x7f566006c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f566850b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566006e2e0 con 0x7f56600d1e20 2024-04-25T01:08:39.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f566850b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566015e510 con 0x7f56600d2200 2024-04-25T01:08:39.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f566850b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f566006ddd0 con 0x7f56600b1010 2024-04-25T01:08:39.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f5665aa6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56600b1010 0x7f566006c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f56662a7640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f56600d1e20 0x7f56600d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33812/0 (socket says 172.21.15.33:33812) 2024-04-25T01:08:39.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f5665aa6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f56600b1010 0x7f566006c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49774/0 (socket says 172.21.15.33:49774) 2024-04-25T01:08:39.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f56662a7640 1 -- 172.21.15.33:0/573544076 learned_addr learned my addr 172.21.15.33:0/573544076 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:39.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f5665aa6640 1 -- 172.21.15.33:0/573544076 >> v1:172.21.15.138:6789/0 conn(0x7f56600d2200 legacy=0x7f56600b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:39.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f5665aa6640 1 -- 172.21.15.33:0/573544076 >> v1:172.21.15.33:6789/0 conn(0x7f56600d1e20 legacy=0x7f56600d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f5665aa6640 1 -- 172.21.15.33:0/573544076 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566006e000 con 0x7f56600b1010 2024-04-25T01:08:39.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f56652a5640 1 -- 172.21.15.33:0/573544076 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3237365939 0 0) 0x7f566006e2e0 con 0x7f56600d1e20 2024-04-25T01:08:39.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f5665aa6640 1 --2- 172.21.15.33:0/573544076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600b1010 0x7f566006c320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf64a3d91325208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f56652a5640 1 -- 172.21.15.33:0/573544076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500b5070 con 0x7f56600b1010 2024-04-25T01:08:39.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f56652a5640 1 -- 172.21.15.33:0/573544076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500a8770 con 0x7f56600b1010 2024-04-25T01:08:39.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f56652a5640 1 -- 172.21.15.33:0/573544076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500a8b40 con 0x7f56600b1010 2024-04-25T01:08:39.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.617+0000 7f566850b640 1 -- 172.21.15.33:0/573544076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600b1010 msgr2=0x7f566006c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 --2- 172.21.15.33:0/573544076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600b1010 0x7f566006c320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001ea70 comp rx=0 tx=0).stop 2024-04-25T01:08:39.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- 172.21.15.33:0/573544076 shutdown_connections 2024-04-25T01:08:39.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 --2- 172.21.15.33:0/573544076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600b1010 0x7f566006c320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- 172.21.15.33:0/573544076 >> 172.21.15.33:0/573544076 conn(0x7f566009a9d0 msgr2=0x7f566009adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:39.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- 172.21.15.33:0/573544076 shutdown_connections 2024-04-25T01:08:39.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- 172.21.15.33:0/573544076 wait complete. 2024-04-25T01:08:39.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 Processor -- start 2024-04-25T01:08:39.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- start start 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56600b1010 0x7f56601f4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600d1e20 0x7f56600cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56600d2200 0x7f56600cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56600d2a00 con 0x7f56600d1e20 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56600d2880 con 0x7f56600d2200 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f56600b51f0 con 0x7f56600b1010 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600d1e20 0x7f56600cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600d1e20 0x7f56600cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49780/0 (socket says 172.21.15.33:49780) 2024-04-25T01:08:39.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 -- 172.21.15.33:0/1681435468 learned_addr learned my addr 172.21.15.33:0/1681435468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 -- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56600b1010 msgr2=0x7f56601f4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56600b1010 0x7f56601f4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 -- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56600d2200 msgr2=0x7f56600cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56600d2200 0x7f56600cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 -- 172.21.15.33:0/1681435468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56600cb240 con 0x7f56600d1e20 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5665aa6640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600d1e20 0x7f56600cd010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5656ffd640 1 -- 172.21.15.33:0/1681435468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500c1070 con 0x7f56600d1e20 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5656ffd640 1 -- 172.21.15.33:0/1681435468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500b6040 con 0x7f56600d1e20 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56600cb520 con 0x7f56600d1e20 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f5656ffd640 1 -- 172.21.15.33:0/1681435468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500b5050 con 0x7f56600d1e20 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56600cbae0 con 0x7f56600d1e20 2024-04-25T01:08:39.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.621+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5660069040 con 0x7f56600d1e20 2024-04-25T01:08:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.625+0000 7f5656ffd640 1 -- 172.21.15.33:0/1681435468 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f56500c4070 con 0x7f56600d1e20 2024-04-25T01:08:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.625+0000 7f5656ffd640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56440678c0 0x7f5644069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.625+0000 7f5656ffd640 1 -- 172.21.15.33:0/1681435468 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f56501231f0 con 0x7f56600d1e20 2024-04-25T01:08:39.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.625+0000 7f56662a7640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56440678c0 0x7f5644069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.625+0000 7f56662a7640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56440678c0 0x7f5644069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f565c0011f0 tx=0x7f565c007930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.629+0000 7f5656ffd640 1 -- 172.21.15.33:0/1681435468 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f566015e510 con 0x7f56600d1e20 2024-04-25T01:08:39.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.781+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5660068540 con 0x7f56600d1e20 2024-04-25T01:08:39.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f5656ffd640 1 -- 172.21.15.33:0/1681435468 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f56500f3a60 con 0x7f56600d1e20 2024-04-25T01:08:39.777 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:39.778 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56440678c0 msgr2=0x7f5644069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56440678c0 0x7f5644069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f565c0011f0 tx=0x7f565c007930 comp rx=0 tx=0).stop 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600d1e20 msgr2=0x7f56600cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600d1e20 0x7f56600cd010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f56500097c0 tx=0x7f565001d2b0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 shutdown_connections 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56440678c0 0x7f5644069d80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56600d2200 0x7f56600cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56600d1e20 0x7f56600cd010 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 --2- 172.21.15.33:0/1681435468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f56600b1010 0x7f56601f4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 >> 172.21.15.33:0/1681435468 conn(0x7f566009a9d0 msgr2=0x7f56600d1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 shutdown_connections 2024-04-25T01:08:39.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.785+0000 7f566850b640 1 -- 172.21.15.33:0/1681435468 wait complete. 2024-04-25T01:08:39.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T01:08:39.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49e14ae640 1 Processor -- start 2024-04-25T01:08:39.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49e14ae640 1 -- start start 2024-04-25T01:08:39.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49e14ae640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f49dc0d26b0 0x7f49dc0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49e14ae640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49dc0d15c0 con 0x7f49dc0d1bc0 2024-04-25T01:08:39.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49e14ae640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49dc0b12e0 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49e14ae640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f49dc0b15c0 con 0x7f49dc0d26b0 2024-04-25T01:08:39.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49daffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f49dc0d1bc0 0x7f49dc0d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33820/0 (socket says 172.21.15.33:33820) 2024-04-25T01:08:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49daffd640 1 -- 172.21.15.33:0/1190929189 learned_addr learned my addr 172.21.15.33:0/1190929189 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49db7fe640 1 --2- 172.21.15.33:0/1190929189 >> v2:172.21.15.33:3300/0 conn(0x7f49dc0d26b0 0x7f49dc0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49d9ffb640 1 -- 172.21.15.33:0/1190929189 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777370096 0 0) 0x7f49dc0d15c0 con 0x7f49dc0d1bc0 2024-04-25T01:08:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49d9ffb640 1 -- 172.21.15.33:0/1190929189 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49c8003530 con 0x7f49dc0d1bc0 2024-04-25T01:08:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49db7fe640 1 -- 172.21.15.33:0/1190929189 >> v1:172.21.15.138:6789/0 conn(0x7f49dc0b0cf0 legacy=0x7f49dc0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49db7fe640 1 -- 172.21.15.33:0/1190929189 >> v1:172.21.15.33:6789/0 conn(0x7f49dc0d1bc0 legacy=0x7f49dc0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.917+0000 7f49db7fe640 1 -- 172.21.15.33:0/1190929189 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc0cfb30 con 0x7f49dc0d26b0 2024-04-25T01:08:39.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49db7fe640 1 --2- 172.21.15.33:0/1190929189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0d26b0 0x7f49dc0b21f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0097c0 tx=0x7f49cc01c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2982a24bd908634 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49d9ffb640 1 -- 172.21.15.33:0/1190929189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49cc0b0070 con 0x7f49dc0d26b0 2024-04-25T01:08:39.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49d9ffb640 1 -- 172.21.15.33:0/1190929189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49cc0ab400 con 0x7f49dc0d26b0 2024-04-25T01:08:39.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49d9ffb640 1 -- 172.21.15.33:0/1190929189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4001a00 con 0x7f49dc0d26b0 2024-04-25T01:08:39.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 -- 172.21.15.33:0/1190929189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0d26b0 msgr2=0x7f49dc0b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 --2- 172.21.15.33:0/1190929189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0d26b0 0x7f49dc0b21f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0097c0 tx=0x7f49cc01c1c0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 -- 172.21.15.33:0/1190929189 shutdown_connections 2024-04-25T01:08:39.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 --2- 172.21.15.33:0/1190929189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0d26b0 0x7f49dc0b21f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 -- 172.21.15.33:0/1190929189 >> 172.21.15.33:0/1190929189 conn(0x7f49dc09a9d0 msgr2=0x7f49dc09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:39.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 -- 172.21.15.33:0/1190929189 shutdown_connections 2024-04-25T01:08:39.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 -- 172.21.15.33:0/1190929189 wait complete. 2024-04-25T01:08:39.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 Processor -- start 2024-04-25T01:08:39.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.921+0000 7f49e14ae640 1 -- start start 2024-04-25T01:08:39.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0b0cf0 0x7f49dc154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49dc0d1bc0 0x7f49dc14d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49dc0d26b0 0x7f49dc15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc15e5a0 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc15e420 con 0x7f49dc0d26b0 2024-04-25T01:08:39.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f49dc16c540 con 0x7f49dc0d1bc0 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0b0cf0 0x7f49dc154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0b0cf0 0x7f49dc154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49794/0 (socket says 172.21.15.33:49794) 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 -- 172.21.15.33:0/497646682 learned_addr learned my addr 172.21.15.33:0/497646682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49da7fc640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49dc0d1bc0 0x7f49dc14d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 -- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49dc0d1bc0 msgr2=0x7f49dc14d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49dc0d1bc0 0x7f49dc14d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 -- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49dc0d26b0 msgr2=0x7f49dc15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49db7fe640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49dc0d26b0 0x7f49dc15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49dc0d26b0 0x7f49dc15a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 -- 172.21.15.33:0/497646682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc15e970 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49daffd640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0b0cf0 0x7f49dc154d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49d400cac0 tx=0x7f49d400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49d37fe640 1 -- 172.21.15.33:0/497646682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d4018070 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49d37fe640 1 -- 172.21.15.33:0/497646682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49d400a820 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc15ec50 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49d37fe640 1 -- 172.21.15.33:0/497646682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d400ac30 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc155270 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49dc069040 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49d37fe640 1 -- 172.21.15.33:0/497646682 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f49d401b070 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49d37fe640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49bc0678c0 0x7f49bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.925+0000 7f49d37fe640 1 -- 172.21.15.33:0/497646682 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f49d4085c70 con 0x7f49dc0b0cf0 2024-04-25T01:08:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.929+0000 7f49da7fc640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49bc0678c0 0x7f49bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:39.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.929+0000 7f49da7fc640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49bc0678c0 0x7f49bc069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49c4007fd0 tx=0x7f49c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:39.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:39.929+0000 7f49d37fe640 1 -- 172.21.15.33:0/497646682 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f49d4056560 con 0x7f49dc0b0cf0 2024-04-25T01:08:40.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.125+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f49dc09aee0 con 0x7f49dc0b0cf0 2024-04-25T01:08:40.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.161+0000 7f49d37fe640 1 -- 172.21.15.33:0/497646682 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f49d4055e60 con 0x7f49dc0b0cf0 2024-04-25T01:08:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49bc0678c0 msgr2=0x7f49bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49bc0678c0 0x7f49bc069d80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49c4007fd0 tx=0x7f49c4009040 comp rx=0 tx=0).stop 2024-04-25T01:08:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0b0cf0 msgr2=0x7f49dc154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:40.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0b0cf0 0x7f49dc154d30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f49d400cac0 tx=0x7f49d400cf80 comp rx=0 tx=0).stop 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 shutdown_connections 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49bc0678c0 0x7f49bc069d80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49dc0d26b0 0x7f49dc15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f49dc0d1bc0 0x7f49dc14d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 --2- 172.21.15.33:0/497646682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49dc0b0cf0 0x7f49dc154d30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 >> 172.21.15.33:0/497646682 conn(0x7f49dc09a9d0 msgr2=0x7f49dc06f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 shutdown_connections 2024-04-25T01:08:40.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.165+0000 7f49e14ae640 1 -- 172.21.15.33:0/497646682 wait complete. 2024-04-25T01:08:40.173 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T01:08:40.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T01:08:40.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe334718640 1 Processor -- start 2024-04-25T01:08:40.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe334718640 1 -- start start 2024-04-25T01:08:40.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe334718640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe32c0d2ac0 0x7fe32c168d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe334718640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe32c0b6cc0 con 0x7fe32c0d1f60 2024-04-25T01:08:40.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe334718640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe32c0b6f30 con 0x7fe32c15e0d0 2024-04-25T01:08:40.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe334718640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe32c0cfb00 con 0x7fe32c0d2ac0 2024-04-25T01:08:40.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe331cb3640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe32c0d1f60 0x7fe32c0d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33832/0 (socket says 172.21.15.33:33832) 2024-04-25T01:08:40.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe331cb3640 1 -- 172.21.15.33:0/473183692 learned_addr learned my addr 172.21.15.33:0/473183692 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:40.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe332cb5640 1 --2- 172.21.15.33:0/473183692 >> v2:172.21.15.138:3300/0 conn(0x7fe32c0d2ac0 0x7fe32c168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3539438606 0 0) 0x7fe32c0b6cc0 con 0x7fe32c0d1f60 2024-04-25T01:08:40.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe318003530 con 0x7fe32c0d1f60 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1810202493 0 0) 0x7fe318003530 con 0x7fe32c0d1f60 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 >> v2:172.21.15.138:3300/0 conn(0x7fe32c0d2ac0 msgr2=0x7fe32c168d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 --2- 172.21.15.33:0/473183692 >> v2:172.21.15.138:3300/0 conn(0x7fe32c0d2ac0 0x7fe32c168d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 >> v1:172.21.15.138:6789/0 conn(0x7fe32c15e0d0 legacy=0x7fe32c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c0b7210 con 0x7fe32c0d1f60 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fe31c001680 con 0x7fe32c0d1f60 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 >> v1:172.21.15.33:6789/0 conn(0x7fe32c0d1f60 legacy=0x7fe32c0d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3180039c0 0x7fe318003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3180043c0 0x7fe318004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe318005090 0x7fe31800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c0cfb00 con 0x7fe3180039c0 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe31800bff0 con 0x7fe318005090 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe31800c1f0 con 0x7fe3180043c0 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3324b4640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3180039c0 0x7fe318003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe332cb5640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe318005090 0x7fe31800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3324b4640 1 -- 172.21.15.33:0/473183692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3180043c0 msgr2=0x7fe318004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3324b4640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe3180043c0 0x7fe318004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3324b4640 1 -- 172.21.15.33:0/473183692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe318005090 msgr2=0x7fe31800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:40.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3324b4640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe318005090 0x7fe31800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe3324b4640 1 -- 172.21.15.33:0/473183692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe318003530 con 0x7fe3180039c0 2024-04-25T01:08:40.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.309+0000 7fe332cb5640 1 -- 172.21.15.33:0/473183692 reap_dead start 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe3324b4640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3180039c0 0x7fe318003da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe328001860 tx=0x7fe32801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2962f0fc17eb4a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3280b0070 con 0x7fe3180039c0 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3280ab600 con 0x7fe3180039c0 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe3314b2640 1 -- 172.21.15.33:0/473183692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328001460 con 0x7fe3180039c0 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 -- 172.21.15.33:0/473183692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3180039c0 msgr2=0x7fe318003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3180039c0 0x7fe318003da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe328001860 tx=0x7fe32801c500 comp rx=0 tx=0).stop 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 -- 172.21.15.33:0/473183692 shutdown_connections 2024-04-25T01:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 --2- 172.21.15.33:0/473183692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3180039c0 0x7fe318003da0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 -- 172.21.15.33:0/473183692 >> 172.21.15.33:0/473183692 conn(0x7fe32c0d3dd0 msgr2=0x7fe32c0d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 -- 172.21.15.33:0/473183692 shutdown_connections 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 -- 172.21.15.33:0/473183692 wait complete. 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 Processor -- start 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.313+0000 7fe334718640 1 -- start start 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe32c15cc70 0x7fe32c1e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c1ec3b0 0x7fe32c1e70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe32c1f7550 0x7fe32c1f79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c0cf570 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe32c0cf3f0 con 0x7fe32c1f7550 2024-04-25T01:08:40.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe32c0cf6f0 con 0x7fe32c15cc70 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c1ec3b0 0x7fe32c1e70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c1ec3b0 0x7fe32c1e70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49804/0 (socket says 172.21.15.33:49804) 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 -- 172.21.15.33:0/1111244271 learned_addr learned my addr 172.21.15.33:0/1111244271 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe332cb5640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe32c1f7550 0x7fe32c1f79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe3324b4640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe32c15cc70 0x7fe32c1e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 -- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe32c15cc70 msgr2=0x7fe32c1e6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe32c15cc70 0x7fe32c1e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 -- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe32c1f7550 msgr2=0x7fe32c1f79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe32c1f7550 0x7fe32c1f79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 -- 172.21.15.33:0/1111244271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32c1eb310 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe332cb5640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe32c1f7550 0x7fe32c1f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe331cb3640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c1ec3b0 0x7fe32c1e70a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c690 tx=0x7fe31c00cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe322ffd640 1 -- 172.21.15.33:0/1111244271 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c018070 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe322ffd640 1 -- 172.21.15.33:0/1111244271 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe31c0072d0 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32c1eb5f0 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe322ffd640 1 -- 172.21.15.33:0/1111244271 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0047f0 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe32c1ebbb0 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe32c069040 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe322ffd640 1 -- 172.21.15.33:0/1111244271 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe31c01b070 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe322ffd640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe3100678f0 0x7fe310069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe322ffd640 1 -- 172.21.15.33:0/1111244271 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe31c0853c0 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.317+0000 7fe3324b4640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe3100678f0 0x7fe310069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.321+0000 7fe3324b4640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe3100678f0 0x7fe310069db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ad000 tx=0x7fe3280a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.321+0000 7fe322ffd640 1 -- 172.21.15.33:0/1111244271 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe31c055c50 con 0x7fe32c1ec3b0 2024-04-25T01:08:40.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:40.517+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe32c0d3600 con 0x7fe32c1ec3b0 2024-04-25T01:08:41.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.177+0000 7fe322ffd640 1 -- 172.21.15.33:0/1111244271 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7fe31c055550 con 0x7fe32c1ec3b0 2024-04-25T01:08:41.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe3100678f0 msgr2=0x7fe310069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe3100678f0 0x7fe310069db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe3280ad000 tx=0x7fe3280a6000 comp rx=0 tx=0).stop 2024-04-25T01:08:41.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c1ec3b0 msgr2=0x7fe32c1e70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c1ec3b0 0x7fe32c1e70a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00c690 tx=0x7fe31c00cb50 comp rx=0 tx=0).stop 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 shutdown_connections 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe3100678f0 0x7fe310069db0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe32c1f7550 0x7fe32c1f79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32c1ec3b0 0x7fe32c1e70a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 --2- 172.21.15.33:0/1111244271 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe32c15cc70 0x7fe32c1e6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 >> 172.21.15.33:0/1111244271 conn(0x7fe32c0d3dd0 msgr2=0x7fe32c0ce9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 shutdown_connections 2024-04-25T01:08:41.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.181+0000 7fe334718640 1 -- 172.21.15.33:0/1111244271 wait complete. 2024-04-25T01:08:41.198 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T01:08:41.198 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.329+0000 7f85d4119640 1 Processor -- start 2024-04-25T01:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.329+0000 7f85d4119640 1 -- start start 2024-04-25T01:08:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d4119640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85cc145470 0x7f85cc15c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d4119640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc0b5530 0x7f85cc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d4119640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85cc15f9a0 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d4119640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f85cc15f750 con 0x7f85cc145470 2024-04-25T01:08:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d4119640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85cc15fc80 con 0x7f85cc0b5530 2024-04-25T01:08:41.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85cc145470 0x7f85cc15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85cc145470 0x7f85cc15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49818/0 (socket says 172.21.15.33:49818) 2024-04-25T01:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 -- 172.21.15.33:0/601439448 learned_addr learned my addr 172.21.15.33:0/601439448 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 -- 172.21.15.33:0/601439448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc0b5530 msgr2=0x7f85cc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d26b6640 1 --2- 172.21.15.33:0/601439448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc0b5530 0x7f85cc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 --2- 172.21.15.33:0/601439448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc0b5530 0x7f85cc0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 -- 172.21.15.33:0/601439448 >> v1:172.21.15.138:6789/0 conn(0x7f85cc0cf0e0 legacy=0x7f85cc0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 -- 172.21.15.33:0/601439448 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc15ff60 con 0x7f85cc145470 2024-04-25T01:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.333+0000 7f85d1eb5640 1 --2- 172.21.15.33:0/601439448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc145470 0x7f85cc15c160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009870 tx=0x7f85c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74d517ead10a6fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d0eb3640 1 -- 172.21.15.33:0/601439448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80aa600 con 0x7f85cc145470 2024-04-25T01:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d0eb3640 1 -- 172.21.15.33:0/601439448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c80af070 con 0x7f85cc145470 2024-04-25T01:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d0eb3640 1 -- 172.21.15.33:0/601439448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c80aa9c0 con 0x7f85cc145470 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- 172.21.15.33:0/601439448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc145470 msgr2=0x7f85cc15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 --2- 172.21.15.33:0/601439448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc145470 0x7f85cc15c160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009870 tx=0x7f85c801c520 comp rx=0 tx=0).stop 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- 172.21.15.33:0/601439448 shutdown_connections 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 --2- 172.21.15.33:0/601439448 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc0b5530 0x7f85cc0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 --2- 172.21.15.33:0/601439448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc145470 0x7f85cc15c160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- 172.21.15.33:0/601439448 >> 172.21.15.33:0/601439448 conn(0x7f85cc0b1810 msgr2=0x7f85cc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- 172.21.15.33:0/601439448 shutdown_connections 2024-04-25T01:08:41.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- 172.21.15.33:0/601439448 wait complete. 2024-04-25T01:08:41.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 Processor -- start 2024-04-25T01:08:41.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- start start 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85cc0b5530 0x7f85cc15dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc0cf0e0 0x7f85cc143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc145470 0x7f85cc1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc168c70 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85cc168af0 con 0x7f85cc0b5530 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85cc168df0 con 0x7f85cc145470 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc0cf0e0 0x7f85cc143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc0cf0e0 0x7f85cc143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49832/0 (socket says 172.21.15.33:49832) 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 -- 172.21.15.33:0/281830632 learned_addr learned my addr 172.21.15.33:0/281830632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d1eb5640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85cc0b5530 0x7f85cc15dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d26b6640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc145470 0x7f85cc1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 -- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc145470 msgr2=0x7f85cc1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc145470 0x7f85cc1443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 -- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85cc0b5530 msgr2=0x7f85cc15dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85cc0b5530 0x7f85cc15dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 -- 172.21.15.33:0/281830632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85cc144a90 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d16b4640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc0cf0e0 0x7f85cc143eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c480 tx=0x7f85bc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85c27fc640 1 -- 172.21.15.33:0/281830632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc012830 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85c27fc640 1 -- 172.21.15.33:0/281830632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85bc017070 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85c27fc640 1 -- 172.21.15.33:0/281830632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85bc012c30 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85cc13be90 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.337+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85cc15e110 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.341+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85cc069040 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.341+0000 7f85c27fc640 1 -- 172.21.15.33:0/281830632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f85bc01c460 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.341+0000 7f85c27fc640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85b00678c0 0x7f85b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.341+0000 7f85c27fc640 1 -- 172.21.15.33:0/281830632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f85bc085e70 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.341+0000 7f85d1eb5640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85b00678c0 0x7f85b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.341+0000 7f85d1eb5640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85b00678c0 0x7f85b0069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ae000 tx=0x7f85c80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.345+0000 7f85c27fc640 1 -- 172.21.15.33:0/281830632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f85bc056660 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.541+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85cc071420 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.541+0000 7f85c27fc640 1 -- 172.21.15.33:0/281830632 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f85bc055f60 con 0x7f85cc0cf0e0 2024-04-25T01:08:41.534 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:41.534 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:08:41.535 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:08:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85b00678c0 msgr2=0x7f85b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85b00678c0 0x7f85b0069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f85c80ae000 tx=0x7f85c80a7000 comp rx=0 tx=0).stop 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc0cf0e0 msgr2=0x7f85cc143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc0cf0e0 0x7f85cc143eb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f85bc00c480 tx=0x7f85bc00c940 comp rx=0 tx=0).stop 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 shutdown_connections 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85b00678c0 0x7f85b0069d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85cc145470 0x7f85cc1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85cc0cf0e0 0x7f85cc143eb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 --2- 172.21.15.33:0/281830632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85cc0b5530 0x7f85cc15dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 >> 172.21.15.33:0/281830632 conn(0x7f85cc0b1810 msgr2=0x7f85cc06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 shutdown_connections 2024-04-25T01:08:41.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.545+0000 7f85d4119640 1 -- 172.21.15.33:0/281830632 wait complete. 2024-04-25T01:08:41.552 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:41.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.669+0000 7fc4f1758640 1 Processor -- start 2024-04-25T01:08:41.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.669+0000 7fc4f1758640 1 -- start start 2024-04-25T01:08:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4f1758640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4ec145470 0x7fc4ec15c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4f1758640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec0cf0e0 0x7fc4ec0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4f1758640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4ec15f9a0 con 0x7fc4ec0cf540 2024-04-25T01:08:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4f1758640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc4ec15f750 con 0x7fc4ec145470 2024-04-25T01:08:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4f1758640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec15fc80 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4ea7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec0cf0e0 0x7fc4ec0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4eaffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4ec145470 0x7fc4ec15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4eb7fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fc4ec0cf540 0x7fc4ec071b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:48500/0 (socket says 172.21.15.33:48500) 2024-04-25T01:08:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4eb7fe640 1 -- 172.21.15.33:0/4120243545 learned_addr learned my addr 172.21.15.33:0/4120243545 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4ea7fc640 1 -- 172.21.15.33:0/4120243545 >> v2:172.21.15.138:3300/0 conn(0x7fc4ec145470 msgr2=0x7fc4ec15c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4ea7fc640 1 --2- 172.21.15.33:0/4120243545 >> v2:172.21.15.138:3300/0 conn(0x7fc4ec145470 0x7fc4ec15c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4ea7fc640 1 -- 172.21.15.33:0/4120243545 >> v1:172.21.15.138:6789/0 conn(0x7fc4ec0cf540 legacy=0x7fc4ec071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4ea7fc640 1 -- 172.21.15.33:0/4120243545 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec15ff60 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.673+0000 7fc4eaffd640 1 --2- 172.21.15.33:0/4120243545 >> v2:172.21.15.138:3300/0 conn(0x7fc4ec145470 0x7fc4ec15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4ea7fc640 1 --2- 172.21.15.33:0/4120243545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec0cf0e0 0x7fc4ec0b4e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00097c0 tx=0x7fc4e001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db7f6b35a6b51140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4e9ffb640 1 -- 172.21.15.33:0/4120243545 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00b1070 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4e9ffb640 1 -- 172.21.15.33:0/4120243545 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4e00ac410 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4e9ffb640 1 -- 172.21.15.33:0/4120243545 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00ac7e0 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 -- 172.21.15.33:0/4120243545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec0cf0e0 msgr2=0x7fc4ec0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 --2- 172.21.15.33:0/4120243545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec0cf0e0 0x7fc4ec0b4e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00097c0 tx=0x7fc4e001c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 -- 172.21.15.33:0/4120243545 shutdown_connections 2024-04-25T01:08:41.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 --2- 172.21.15.33:0/4120243545 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec0cf0e0 0x7fc4ec0b4e20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 --2- 172.21.15.33:0/4120243545 >> v2:172.21.15.138:3300/0 conn(0x7fc4ec145470 0x7fc4ec15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 -- 172.21.15.33:0/4120243545 >> 172.21.15.33:0/4120243545 conn(0x7fc4ec0b1810 msgr2=0x7fc4ec0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 -- 172.21.15.33:0/4120243545 shutdown_connections 2024-04-25T01:08:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 -- 172.21.15.33:0/4120243545 wait complete. 2024-04-25T01:08:41.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 Processor -- start 2024-04-25T01:08:41.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.677+0000 7fc4f1758640 1 -- start start 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0cf0e0 0x7fc4ec1f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec0cf540 0x7fc4ec15b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec145470 0x7fc4ec159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec168c70 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4ec168af0 con 0x7fc4ec0cf540 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4ec168df0 con 0x7fc4ec145470 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0cf0e0 0x7fc4ec1f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0cf0e0 0x7fc4ec1f5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49842/0 (socket says 172.21.15.33:49842) 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 -- 172.21.15.33:0/441527093 learned_addr learned my addr 172.21.15.33:0/441527093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:41.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4ea7fc640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec0cf540 0x7fc4ec15b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eb7fe640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec145470 0x7fc4ec159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 -- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec145470 msgr2=0x7fc4ec159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec145470 0x7fc4ec159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 -- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec0cf540 msgr2=0x7fc4ec15b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec0cf540 0x7fc4ec15b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 -- 172.21.15.33:0/441527093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4ec142950 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eb7fe640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec145470 0x7fc4ec159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4eaffd640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0cf0e0 0x7fc4ec1f5000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00ebe0 tx=0x7fc4dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4d37fe640 1 -- 172.21.15.33:0/441527093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc017070 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4d37fe640 1 -- 172.21.15.33:0/441527093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4dc00cc30 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4ec142c30 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4d37fe640 1 -- 172.21.15.33:0/441527093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc013470 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4ec1431f0 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4ec069040 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4d37fe640 1 -- 172.21.15.33:0/441527093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc4dc01a070 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4d37fe640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4d80678f0 0x7fc4d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4d37fe640 1 -- 172.21.15.33:0/441527093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc4dc086350 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.681+0000 7fc4ea7fc640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4d80678f0 0x7fc4d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.685+0000 7fc4ea7fc640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4d80678f0 0x7fc4d8069db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00ae000 tx=0x7fc4e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.685+0000 7fc4d37fe640 1 -- 172.21.15.33:0/441527093 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4dc0563e0 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.845+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4ec0b20e0 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4d37fe640 1 -- 172.21.15.33:0/441527093 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fc4dc05a6e0 con 0x7fc4ec0cf0e0 2024-04-25T01:08:41.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:41.842 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:41.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4d80678f0 msgr2=0x7fc4d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4d80678f0 0x7fc4d8069db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4e00ae000 tx=0x7fc4e0008040 comp rx=0 tx=0).stop 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0cf0e0 msgr2=0x7fc4ec1f5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0cf0e0 0x7fc4ec1f5000 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00ebe0 tx=0x7fc4dc00c520 comp rx=0 tx=0).stop 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 shutdown_connections 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4d80678f0 0x7fc4d8069db0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4ec145470 0x7fc4ec159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ec0cf540 0x7fc4ec15b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 --2- 172.21.15.33:0/441527093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ec0cf0e0 0x7fc4ec1f5000 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 >> 172.21.15.33:0/441527093 conn(0x7fc4ec0b1810 msgr2=0x7fc4ec071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 shutdown_connections 2024-04-25T01:08:41.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.849+0000 7fc4f1758640 1 -- 172.21.15.33:0/441527093 wait complete. 2024-04-25T01:08:41.860 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T01:08:41.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:08:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d7b455640 1 Processor -- start 2024-04-25T01:08:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d7b455640 1 -- start start 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d7b455640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3d7415c2d0 0x7f3d7415c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d7b455640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3d7406b210 0x7f3d7406b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d7b455640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d740d0a00 con 0x7f3d740d0620 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d7b455640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7406bbc0 con 0x7f3d7406b210 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d7b455640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3d7406e3a0 con 0x7f3d7415c2d0 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d799f2640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3d7406b210 0x7f3d7406b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d799f2640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3d7406b210 0x7f3d7406b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49846/0 (socket says 172.21.15.33:49846) 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d799f2640 1 -- 172.21.15.33:0/3974946007 learned_addr learned my addr 172.21.15.33:0/3974946007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d789f0640 1 --2- 172.21.15.33:0/3974946007 >> v2:172.21.15.138:3300/0 conn(0x7f3d7415c2d0 0x7f3d7415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d799f2640 1 -- 172.21.15.33:0/3974946007 >> v2:172.21.15.138:3300/0 conn(0x7f3d7415c2d0 msgr2=0x7f3d7415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:41.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d799f2640 1 --2- 172.21.15.33:0/3974946007 >> v2:172.21.15.138:3300/0 conn(0x7f3d7415c2d0 0x7f3d7415c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d799f2640 1 -- 172.21.15.33:0/3974946007 >> v1:172.21.15.138:6789/0 conn(0x7f3d740d0620 legacy=0x7f3d7406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.993+0000 7f3d799f2640 1 -- 172.21.15.33:0/3974946007 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7415cdc0 con 0x7f3d7406b210 2024-04-25T01:08:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 --2- 172.21.15.33:0/3974946007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7406b210 0x7f3d7406b680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d740aecf0 tx=0x7f3d6c01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576b6848bb6ae05f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d6bfff640 1 -- 172.21.15.33:0/3974946007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0a5840 con 0x7f3d7406b210 2024-04-25T01:08:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d6bfff640 1 -- 172.21.15.33:0/3974946007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6c0a59e0 con 0x7f3d7406b210 2024-04-25T01:08:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d6bfff640 1 -- 172.21.15.33:0/3974946007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0a5db0 con 0x7f3d7406b210 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- 172.21.15.33:0/3974946007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7406b210 msgr2=0x7f3d7406b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3974946007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7406b210 0x7f3d7406b680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3d740aecf0 tx=0x7f3d6c01c980 comp rx=0 tx=0).stop 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- 172.21.15.33:0/3974946007 shutdown_connections 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3974946007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7406b210 0x7f3d7406b680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3974946007 >> v2:172.21.15.138:3300/0 conn(0x7f3d7415c2d0 0x7f3d7415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- 172.21.15.33:0/3974946007 >> 172.21.15.33:0/3974946007 conn(0x7f3d740b29c0 msgr2=0x7f3d740b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- 172.21.15.33:0/3974946007 shutdown_connections 2024-04-25T01:08:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- 172.21.15.33:0/3974946007 wait complete. 2024-04-25T01:08:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 Processor -- start 2024-04-25T01:08:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- start start 2024-04-25T01:08:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7406b210 0x7f3d7414a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d740d0620 0x7f3d741f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7415c2d0 0x7f3d74141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d7415dca0 con 0x7f3d7415c2d0 2024-04-25T01:08:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d740adf30 con 0x7f3d7406b210 2024-04-25T01:08:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d7b455640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d7415de20 con 0x7f3d740d0620 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7415c2d0 0x7f3d74141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7415c2d0 0x7f3d74141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49862/0 (socket says 172.21.15.33:49862) 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 -- 172.21.15.33:0/3035728730 learned_addr learned my addr 172.21.15.33:0/3035728730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d789f0640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d740d0620 0x7f3d741f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d791f1640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7406b210 0x7f3d7414a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 -- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d740d0620 msgr2=0x7f3d741f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d740d0620 0x7f3d741f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 -- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7406b210 msgr2=0x7f3d7414a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7406b210 0x7f3d7414a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:41.997+0000 7f3d799f2640 1 -- 172.21.15.33:0/3035728730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d7414b050 con 0x7f3d7415c2d0 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d799f2640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7415c2d0 0x7f3d74141e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c01a5b0 tx=0x7f3d6c019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d69ffb640 1 -- 172.21.15.33:0/3035728730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0bc020 con 0x7f3d7415c2d0 2024-04-25T01:08:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d69ffb640 1 -- 172.21.15.33:0/3035728730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d6c0b86b0 con 0x7f3d7415c2d0 2024-04-25T01:08:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d7414b330 con 0x7f3d7415c2d0 2024-04-25T01:08:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d69ffb640 1 -- 172.21.15.33:0/3035728730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0b8aa0 con 0x7f3d7415c2d0 2024-04-25T01:08:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d741465b0 con 0x7f3d7415c2d0 2024-04-25T01:08:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d74069040 con 0x7f3d7415c2d0 2024-04-25T01:08:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d69ffb640 1 -- 172.21.15.33:0/3035728730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d6c0bf020 con 0x7f3d7415c2d0 2024-04-25T01:08:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d69ffb640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d580678f0 0x7f3d58069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d69ffb640 1 -- 172.21.15.33:0/3035728730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3d6c120c70 con 0x7f3d7415c2d0 2024-04-25T01:08:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d791f1640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d580678f0 0x7f3d58069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:41.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.001+0000 7f3d791f1640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d580678f0 0x7f3d58069db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d7414bf60 tx=0x7f3d70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:41.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.005+0000 7f3d69ffb640 1 -- 172.21.15.33:0/3035728730 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3d6c0165a0 con 0x7f3d7415c2d0 2024-04-25T01:08:42.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.161+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3d7406f200 con 0x7f3d580678f0 2024-04-25T01:08:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.189+0000 7f3d69ffb640 1 -- 172.21.15.33:0/3035728730 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332570 (secure 0 0 0) 0x7f3d7406f200 con 0x7f3d580678f0 2024-04-25T01:08:42.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:42.188 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d580678f0 msgr2=0x7f3d58069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d580678f0 0x7f3d58069db0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3d7414bf60 tx=0x7f3d70009040 comp rx=0 tx=0).stop 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7415c2d0 msgr2=0x7f3d74141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7415c2d0 0x7f3d74141e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c01a5b0 tx=0x7f3d6c019720 comp rx=0 tx=0).stop 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 shutdown_connections 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d580678f0 0x7f3d58069db0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d7415c2d0 0x7f3d74141e40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d740d0620 0x7f3d741f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 --2- 172.21.15.33:0/3035728730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d7406b210 0x7f3d7414a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 >> 172.21.15.33:0/3035728730 conn(0x7f3d740b29c0 msgr2=0x7f3d740b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 shutdown_connections 2024-04-25T01:08:42.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.193+0000 7f3d7b455640 1 -- 172.21.15.33:0/3035728730 wait complete. 2024-04-25T01:08:42.206 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-04-25T01:08:41.091571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223588,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751132,"statfs":{"total":773094113280,"available":772865159168,"internally_reserved":0,"allocated":3317760,"data_stored":2009438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.585087"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830172+0000","last_change":"2024-04-25T01:08:23.464284+0000","last_active":"2024-04-25T01:08:29.830172+0000","last_peered":"2024-04-25T01:08:29.830172+0000","last_clean":"2024-04-25T01:08:29.830172+0000","last_became_active":"2024-04-25T01:08:23.464018+0000","last_became_peered":"2024-04-25T01:08:23.464018+0000","last_unstale":"2024-04-25T01:08:29.830172+0000","last_undegraded":"2024-04-25T01:08:29.830172+0000","last_fullsized":"2024-04-25T01:08:29.830172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:27.195443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509770+0000","last_change":"2024-04-25T01:08:24.479420+0000","last_active":"2024-04-25T01:08:29.509770+0000","last_peered":"2024-04-25T01:08:29.509770+0000","last_clean":"2024-04-25T01:08:29.509770+0000","last_became_active":"2024-04-25T01:08:24.479003+0000","last_became_peered":"2024-04-25T01:08:24.479003+0000","last_unstale":"2024-04-25T01:08:29.509770+0000","last_undegraded":"2024-04-25T01:08:29.509770+0000","last_fullsized":"2024-04-25T01:08:29.509770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:10.309810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507734+0000","last_change":"2024-04-25T01:08:28.865743+0000","last_active":"2024-04-25T01:08:29.507734+0000","last_peered":"2024-04-25T01:08:29.507734+0000","last_clean":"2024-04-25T01:08:29.507734+0000","last_became_active":"2024-04-25T01:08:28.864434+0000","last_became_peered":"2024-04-25T01:08:28.864434+0000","last_unstale":"2024-04-25T01:08:29.507734+0000","last_undegraded":"2024-04-25T01:08:29.507734+0000","last_fullsized":"2024-04-25T01:08:29.507734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.379931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510114+0000","last_change":"2024-04-25T01:08:23.447876+0000","last_active":"2024-04-25T01:08:29.510114+0000","last_peered":"2024-04-25T01:08:29.510114+0000","last_clean":"2024-04-25T01:08:29.510114+0000","last_became_active":"2024-04-25T01:08:23.447588+0000","last_became_peered":"2024-04-25T01:08:23.447588+0000","last_unstale":"2024-04-25T01:08:29.510114+0000","last_undegraded":"2024-04-25T01:08:29.510114+0000","last_fullsized":"2024-04-25T01:08:29.510114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830021+0000","last_change":"2024-04-25T01:08:24.494422+0000","last_active":"2024-04-25T01:08:29.830021+0000","last_peered":"2024-04-25T01:08:29.830021+0000","last_clean":"2024-04-25T01:08:29.830021+0000","last_became_active":"2024-04-25T01:08:24.494035+0000","last_became_peered":"2024-04-25T01:08:24.494035+0000","last_unstale":"2024-04-25T01:08:29.830021+0000","last_undegraded":"2024-04-25T01:08:29.830021+0000","last_fullsized":"2024-04-25T01:08:29.830021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510126+0000","last_change":"2024-04-25T01:08:28.548777+0000","last_active":"2024-04-25T01:08:29.510126+0000","last_peered":"2024-04-25T01:08:29.510126+0000","last_clean":"2024-04-25T01:08:29.510126+0000","last_became_active":"2024-04-25T01:08:28.545213+0000","last_became_peered":"2024-04-25T01:08:28.545213+0000","last_unstale":"2024-04-25T01:08:29.510126+0000","last_undegraded":"2024-04-25T01:08:29.510126+0000","last_fullsized":"2024-04-25T01:08:29.510126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:14.372448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826296+0000","last_change":"2024-04-25T01:08:23.483744+0000","last_active":"2024-04-25T01:08:29.826296+0000","last_peered":"2024-04-25T01:08:29.826296+0000","last_clean":"2024-04-25T01:08:29.826296+0000","last_became_active":"2024-04-25T01:08:23.483245+0000","last_became_peered":"2024-04-25T01:08:23.483245+0000","last_unstale":"2024-04-25T01:08:29.826296+0000","last_undegraded":"2024-04-25T01:08:29.826296+0000","last_fullsized":"2024-04-25T01:08:29.826296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:38.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826404+0000","last_change":"2024-04-25T01:08:24.480206+0000","last_active":"2024-04-25T01:08:29.826404+0000","last_peered":"2024-04-25T01:08:29.826404+0000","last_clean":"2024-04-25T01:08:29.826404+0000","last_became_active":"2024-04-25T01:08:24.479327+0000","last_became_peered":"2024-04-25T01:08:24.479327+0000","last_unstale":"2024-04-25T01:08:29.826404+0000","last_undegraded":"2024-04-25T01:08:29.826404+0000","last_fullsized":"2024-04-25T01:08:29.826404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.885275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510631+0000","last_change":"2024-04-25T01:08:28.545701+0000","last_active":"2024-04-25T01:08:29.510631+0000","last_peered":"2024-04-25T01:08:29.510631+0000","last_clean":"2024-04-25T01:08:29.510631+0000","last_became_active":"2024-04-25T01:08:28.544731+0000","last_became_peered":"2024-04-25T01:08:28.544731+0000","last_unstale":"2024-04-25T01:08:29.510631+0000","last_undegraded":"2024-04-25T01:08:29.510631+0000","last_fullsized":"2024-04-25T01:08:29.510631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.483124+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.482403+0000","last_became_peered":"2024-04-25T01:08:23.482403+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:22.095395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827570+0000","last_change":"2024-04-25T01:08:24.488697+0000","last_active":"2024-04-25T01:08:29.827570+0000","last_peered":"2024-04-25T01:08:29.827570+0000","last_clean":"2024-04-25T01:08:29.827570+0000","last_became_active":"2024-04-25T01:08:24.487850+0000","last_became_peered":"2024-04-25T01:08:24.487850+0000","last_unstale":"2024-04-25T01:08:29.827570+0000","last_undegraded":"2024-04-25T01:08:29.827570+0000","last_fullsized":"2024-04-25T01:08:29.827570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:57.185281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510492+0000","last_change":"2024-04-25T01:08:28.545916+0000","last_active":"2024-04-25T01:08:29.510492+0000","last_peered":"2024-04-25T01:08:29.510492+0000","last_clean":"2024-04-25T01:08:29.510492+0000","last_became_active":"2024-04-25T01:08:28.544666+0000","last_became_peered":"2024-04-25T01:08:28.544666+0000","last_unstale":"2024-04-25T01:08:29.510492+0000","last_undegraded":"2024-04-25T01:08:29.510492+0000","last_fullsized":"2024-04-25T01:08:29.510492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:03.657733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827605+0000","last_change":"2024-04-25T01:08:24.498408+0000","last_active":"2024-04-25T01:08:29.827605+0000","last_peered":"2024-04-25T01:08:29.827605+0000","last_clean":"2024-04-25T01:08:29.827605+0000","last_became_active":"2024-04-25T01:08:24.497888+0000","last_became_peered":"2024-04-25T01:08:24.497888+0000","last_unstale":"2024-04-25T01:08:29.827605+0000","last_undegraded":"2024-04-25T01:08:29.827605+0000","last_fullsized":"2024-04-25T01:08:29.827605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828026+0000","last_change":"2024-04-25T01:08:23.482207+0000","last_active":"2024-04-25T01:08:29.828026+0000","last_peered":"2024-04-25T01:08:29.828026+0000","last_clean":"2024-04-25T01:08:29.828026+0000","last_became_active":"2024-04-25T01:08:23.475806+0000","last_became_peered":"2024-04-25T01:08:23.475806+0000","last_unstale":"2024-04-25T01:08:29.828026+0000","last_undegraded":"2024-04-25T01:08:29.828026+0000","last_fullsized":"2024-04-25T01:08:29.828026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:56.775800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512324+0000","last_change":"2024-04-25T01:08:28.537642+0000","last_active":"2024-04-25T01:08:29.512324+0000","last_peered":"2024-04-25T01:08:29.512324+0000","last_clean":"2024-04-25T01:08:29.512324+0000","last_became_active":"2024-04-25T01:08:28.536434+0000","last_became_peered":"2024-04-25T01:08:28.536434+0000","last_unstale":"2024-04-25T01:08:29.512324+0000","last_undegraded":"2024-04-25T01:08:29.512324+0000","last_fullsized":"2024-04-25T01:08:29.512324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508548+0000","last_change":"2024-04-25T01:08:24.497990+0000","last_active":"2024-04-25T01:08:29.508548+0000","last_peered":"2024-04-25T01:08:29.508548+0000","last_clean":"2024-04-25T01:08:29.508548+0000","last_became_active":"2024-04-25T01:08:24.497701+0000","last_became_peered":"2024-04-25T01:08:24.497701+0000","last_unstale":"2024-04-25T01:08:29.508548+0000","last_undegraded":"2024-04-25T01:08:29.508548+0000","last_fullsized":"2024-04-25T01:08:29.508548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:25.853831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.448779+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.448310+0000","last_became_peered":"2024-04-25T01:08:23.448310+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510707+0000","last_change":"2024-04-25T01:08:28.546163+0000","last_active":"2024-04-25T01:08:29.510707+0000","last_peered":"2024-04-25T01:08:29.510707+0000","last_clean":"2024-04-25T01:08:29.510707+0000","last_became_active":"2024-04-25T01:08:28.545022+0000","last_became_peered":"2024-04-25T01:08:28.545022+0000","last_unstale":"2024-04-25T01:08:29.510707+0000","last_undegraded":"2024-04-25T01:08:29.510707+0000","last_fullsized":"2024-04-25T01:08:29.510707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508535+0000","last_change":"2024-04-25T01:08:24.472427+0000","last_active":"2024-04-25T01:08:29.508535+0000","last_peered":"2024-04-25T01:08:29.508535+0000","last_clean":"2024-04-25T01:08:29.508535+0000","last_became_active":"2024-04-25T01:08:24.471579+0000","last_became_peered":"2024-04-25T01:08:24.471579+0000","last_unstale":"2024-04-25T01:08:29.508535+0000","last_undegraded":"2024-04-25T01:08:29.508535+0000","last_fullsized":"2024-04-25T01:08:29.508535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828376+0000","last_change":"2024-04-25T01:08:23.475602+0000","last_active":"2024-04-25T01:08:29.828376+0000","last_peered":"2024-04-25T01:08:29.828376+0000","last_clean":"2024-04-25T01:08:29.828376+0000","last_became_active":"2024-04-25T01:08:23.475273+0000","last_became_peered":"2024-04-25T01:08:23.475273+0000","last_unstale":"2024-04-25T01:08:29.828376+0000","last_undegraded":"2024-04-25T01:08:29.828376+0000","last_fullsized":"2024-04-25T01:08:29.828376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830145+0000","last_change":"2024-04-25T01:08:28.858908+0000","last_active":"2024-04-25T01:08:29.830145+0000","last_peered":"2024-04-25T01:08:29.830145+0000","last_clean":"2024-04-25T01:08:29.830145+0000","last_became_active":"2024-04-25T01:08:28.858052+0000","last_became_peered":"2024-04-25T01:08:28.858052+0000","last_unstale":"2024-04-25T01:08:29.830145+0000","last_undegraded":"2024-04-25T01:08:29.830145+0000","last_fullsized":"2024-04-25T01:08:29.830145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:53.339121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511155+0000","last_change":"2024-04-25T01:08:24.473202+0000","last_active":"2024-04-25T01:08:29.511155+0000","last_peered":"2024-04-25T01:08:29.511155+0000","last_clean":"2024-04-25T01:08:29.511155+0000","last_became_active":"2024-04-25T01:08:24.470580+0000","last_became_peered":"2024-04-25T01:08:24.470580+0000","last_unstale":"2024-04-25T01:08:29.511155+0000","last_undegraded":"2024-04-25T01:08:29.511155+0000","last_fullsized":"2024-04-25T01:08:29.511155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:27.238691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828899+0000","last_change":"2024-04-25T01:08:23.482632+0000","last_active":"2024-04-25T01:08:29.828899+0000","last_peered":"2024-04-25T01:08:29.828899+0000","last_clean":"2024-04-25T01:08:29.828899+0000","last_became_active":"2024-04-25T01:08:23.482050+0000","last_became_peered":"2024-04-25T01:08:23.482050+0000","last_unstale":"2024-04-25T01:08:29.828899+0000","last_undegraded":"2024-04-25T01:08:29.828899+0000","last_fullsized":"2024-04-25T01:08:29.828899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512530+0000","last_change":"2024-04-25T01:08:28.538805+0000","last_active":"2024-04-25T01:08:29.512530+0000","last_peered":"2024-04-25T01:08:29.512530+0000","last_clean":"2024-04-25T01:08:29.512530+0000","last_became_active":"2024-04-25T01:08:28.537512+0000","last_became_peered":"2024-04-25T01:08:28.537512+0000","last_unstale":"2024-04-25T01:08:29.512530+0000","last_undegraded":"2024-04-25T01:08:29.512530+0000","last_fullsized":"2024-04-25T01:08:29.512530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829865+0000","last_change":"2024-04-25T01:08:24.488971+0000","last_active":"2024-04-25T01:08:29.829865+0000","last_peered":"2024-04-25T01:08:29.829865+0000","last_clean":"2024-04-25T01:08:29.829865+0000","last_became_active":"2024-04-25T01:08:24.488318+0000","last_became_peered":"2024-04-25T01:08:24.488318+0000","last_unstale":"2024-04-25T01:08:29.829865+0000","last_undegraded":"2024-04-25T01:08:29.829865+0000","last_fullsized":"2024-04-25T01:08:29.829865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:48.906624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829517+0000","last_change":"2024-04-25T01:08:23.484304+0000","last_active":"2024-04-25T01:08:29.829517+0000","last_peered":"2024-04-25T01:08:29.829517+0000","last_clean":"2024-04-25T01:08:29.829517+0000","last_became_active":"2024-04-25T01:08:23.483396+0000","last_became_peered":"2024-04-25T01:08:23.483396+0000","last_unstale":"2024-04-25T01:08:29.829517+0000","last_undegraded":"2024-04-25T01:08:29.829517+0000","last_fullsized":"2024-04-25T01:08:29.829517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510293+0000","last_change":"2024-04-25T01:08:28.861041+0000","last_active":"2024-04-25T01:08:29.510293+0000","last_peered":"2024-04-25T01:08:29.510293+0000","last_clean":"2024-04-25T01:08:29.510293+0000","last_became_active":"2024-04-25T01:08:28.859853+0000","last_became_peered":"2024-04-25T01:08:28.859853+0000","last_unstale":"2024-04-25T01:08:29.510293+0000","last_undegraded":"2024-04-25T01:08:29.510293+0000","last_fullsized":"2024-04-25T01:08:29.510293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:58.665747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510472+0000","last_change":"2024-04-25T01:08:24.513917+0000","last_active":"2024-04-25T01:08:29.510472+0000","last_peered":"2024-04-25T01:08:29.510472+0000","last_clean":"2024-04-25T01:08:29.510472+0000","last_became_active":"2024-04-25T01:08:24.513551+0000","last_became_peered":"2024-04-25T01:08:24.513551+0000","last_unstale":"2024-04-25T01:08:29.510472+0000","last_undegraded":"2024-04-25T01:08:29.510472+0000","last_fullsized":"2024-04-25T01:08:29.510472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829765+0000","last_change":"2024-04-25T01:08:23.463105+0000","last_active":"2024-04-25T01:08:29.829765+0000","last_peered":"2024-04-25T01:08:29.829765+0000","last_clean":"2024-04-25T01:08:29.829765+0000","last_became_active":"2024-04-25T01:08:23.462766+0000","last_became_peered":"2024-04-25T01:08:23.462766+0000","last_unstale":"2024-04-25T01:08:29.829765+0000","last_undegraded":"2024-04-25T01:08:29.829765+0000","last_fullsized":"2024-04-25T01:08:29.829765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507606+0000","last_change":"2024-04-25T01:08:28.859248+0000","last_active":"2024-04-25T01:08:29.507606+0000","last_peered":"2024-04-25T01:08:29.507606+0000","last_clean":"2024-04-25T01:08:29.507606+0000","last_became_active":"2024-04-25T01:08:28.858134+0000","last_became_peered":"2024-04-25T01:08:28.858134+0000","last_unstale":"2024-04-25T01:08:29.507606+0000","last_undegraded":"2024-04-25T01:08:29.507606+0000","last_fullsized":"2024-04-25T01:08:29.507606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:54.239410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827142+0000","last_change":"2024-04-25T01:08:24.463169+0000","last_active":"2024-04-25T01:08:29.827142+0000","last_peered":"2024-04-25T01:08:29.827142+0000","last_clean":"2024-04-25T01:08:29.827142+0000","last_became_active":"2024-04-25T01:08:24.462836+0000","last_became_peered":"2024-04-25T01:08:24.462836+0000","last_unstale":"2024-04-25T01:08:29.827142+0000","last_undegraded":"2024-04-25T01:08:29.827142+0000","last_fullsized":"2024-04-25T01:08:29.827142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826900+0000","last_change":"2024-04-25T01:08:23.482862+0000","last_active":"2024-04-25T01:08:29.826900+0000","last_peered":"2024-04-25T01:08:29.826900+0000","last_clean":"2024-04-25T01:08:29.826900+0000","last_became_active":"2024-04-25T01:08:23.482148+0000","last_became_peered":"2024-04-25T01:08:23.482148+0000","last_unstale":"2024-04-25T01:08:29.826900+0000","last_undegraded":"2024-04-25T01:08:29.826900+0000","last_fullsized":"2024-04-25T01:08:29.826900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:38.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826923+0000","last_change":"2024-04-25T01:08:28.853689+0000","last_active":"2024-04-25T01:08:29.826923+0000","last_peered":"2024-04-25T01:08:29.826923+0000","last_clean":"2024-04-25T01:08:29.826923+0000","last_became_active":"2024-04-25T01:08:28.852691+0000","last_became_peered":"2024-04-25T01:08:28.852691+0000","last_unstale":"2024-04-25T01:08:29.826923+0000","last_undegraded":"2024-04-25T01:08:29.826923+0000","last_fullsized":"2024-04-25T01:08:29.826923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:01.247919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828543+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.828543+0000","last_peered":"2024-04-25T01:08:29.828543+0000","last_clean":"2024-04-25T01:08:29.828543+0000","last_became_active":"2024-04-25T01:08:24.489833+0000","last_became_peered":"2024-04-25T01:08:24.489833+0000","last_unstale":"2024-04-25T01:08:29.828543+0000","last_undegraded":"2024-04-25T01:08:29.828543+0000","last_fullsized":"2024-04-25T01:08:29.828543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:43.847545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:23.451486+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:23.451147+0000","last_became_peered":"2024-04-25T01:08:23.451147+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824912+0000","last_change":"2024-04-25T01:08:28.546785+0000","last_active":"2024-04-25T01:08:29.824912+0000","last_peered":"2024-04-25T01:08:29.824912+0000","last_clean":"2024-04-25T01:08:29.824912+0000","last_became_active":"2024-04-25T01:08:28.546088+0000","last_became_peered":"2024-04-25T01:08:28.546088+0000","last_unstale":"2024-04-25T01:08:29.824912+0000","last_undegraded":"2024-04-25T01:08:29.824912+0000","last_fullsized":"2024-04-25T01:08:29.824912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511711+0000","last_change":"2024-04-25T01:08:24.469958+0000","last_active":"2024-04-25T01:08:29.511711+0000","last_peered":"2024-04-25T01:08:29.511711+0000","last_clean":"2024-04-25T01:08:29.511711+0000","last_became_active":"2024-04-25T01:08:24.468421+0000","last_became_peered":"2024-04-25T01:08:24.468421+0000","last_unstale":"2024-04-25T01:08:29.511711+0000","last_undegraded":"2024-04-25T01:08:29.511711+0000","last_fullsized":"2024-04-25T01:08:29.511711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:37.314427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826336+0000","last_change":"2024-04-25T01:08:23.485610+0000","last_active":"2024-04-25T01:08:29.826336+0000","last_peered":"2024-04-25T01:08:29.826336+0000","last_clean":"2024-04-25T01:08:29.826336+0000","last_became_active":"2024-04-25T01:08:23.484885+0000","last_became_peered":"2024-04-25T01:08:23.484885+0000","last_unstale":"2024-04-25T01:08:29.826336+0000","last_undegraded":"2024-04-25T01:08:29.826336+0000","last_fullsized":"2024-04-25T01:08:29.826336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:41.214012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826899+0000","last_change":"2024-04-25T01:08:28.541717+0000","last_active":"2024-04-25T01:08:29.826899+0000","last_peered":"2024-04-25T01:08:29.826899+0000","last_clean":"2024-04-25T01:08:29.826899+0000","last_became_active":"2024-04-25T01:08:28.541136+0000","last_became_peered":"2024-04-25T01:08:28.541136+0000","last_unstale":"2024-04-25T01:08:29.826899+0000","last_undegraded":"2024-04-25T01:08:29.826899+0000","last_fullsized":"2024-04-25T01:08:29.826899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824739+0000","last_change":"2024-04-25T01:08:24.468504+0000","last_active":"2024-04-25T01:08:29.824739+0000","last_peered":"2024-04-25T01:08:29.824739+0000","last_clean":"2024-04-25T01:08:29.824739+0000","last_became_active":"2024-04-25T01:08:24.468220+0000","last_became_peered":"2024-04-25T01:08:24.468220+0000","last_unstale":"2024-04-25T01:08:29.824739+0000","last_undegraded":"2024-04-25T01:08:29.824739+0000","last_fullsized":"2024-04-25T01:08:29.824739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825079+0000","last_change":"2024-04-25T01:08:23.448344+0000","last_active":"2024-04-25T01:08:29.825079+0000","last_peered":"2024-04-25T01:08:29.825079+0000","last_clean":"2024-04-25T01:08:29.825079+0000","last_became_active":"2024-04-25T01:08:23.447926+0000","last_became_peered":"2024-04-25T01:08:23.447926+0000","last_unstale":"2024-04-25T01:08:29.825079+0000","last_undegraded":"2024-04-25T01:08:29.825079+0000","last_fullsized":"2024-04-25T01:08:29.825079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830344+0000","last_change":"2024-04-25T01:08:28.547149+0000","last_active":"2024-04-25T01:08:29.830344+0000","last_peered":"2024-04-25T01:08:29.830344+0000","last_clean":"2024-04-25T01:08:29.830344+0000","last_became_active":"2024-04-25T01:08:28.546228+0000","last_became_peered":"2024-04-25T01:08:28.546228+0000","last_unstale":"2024-04-25T01:08:29.830344+0000","last_undegraded":"2024-04-25T01:08:29.830344+0000","last_fullsized":"2024-04-25T01:08:29.830344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.235046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825749+0000","last_change":"2024-04-25T01:08:24.458889+0000","last_active":"2024-04-25T01:08:29.825749+0000","last_peered":"2024-04-25T01:08:29.825749+0000","last_clean":"2024-04-25T01:08:29.825749+0000","last_became_active":"2024-04-25T01:08:24.458591+0000","last_became_peered":"2024-04-25T01:08:24.458591+0000","last_unstale":"2024-04-25T01:08:29.825749+0000","last_undegraded":"2024-04-25T01:08:29.825749+0000","last_fullsized":"2024-04-25T01:08:29.825749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:40.997594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829116+0000","last_change":"2024-04-25T01:08:23.481327+0000","last_active":"2024-04-25T01:08:29.829116+0000","last_peered":"2024-04-25T01:08:29.829116+0000","last_clean":"2024-04-25T01:08:29.829116+0000","last_became_active":"2024-04-25T01:08:23.475537+0000","last_became_peered":"2024-04-25T01:08:23.475537+0000","last_unstale":"2024-04-25T01:08:29.829116+0000","last_undegraded":"2024-04-25T01:08:29.829116+0000","last_fullsized":"2024-04-25T01:08:29.829116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829514+0000","last_change":"2024-04-25T01:08:28.859351+0000","last_active":"2024-04-25T01:08:29.829514+0000","last_peered":"2024-04-25T01:08:29.829514+0000","last_clean":"2024-04-25T01:08:29.829514+0000","last_became_active":"2024-04-25T01:08:28.858099+0000","last_became_peered":"2024-04-25T01:08:28.858099+0000","last_unstale":"2024-04-25T01:08:29.829514+0000","last_undegraded":"2024-04-25T01:08:29.829514+0000","last_fullsized":"2024-04-25T01:08:29.829514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:46.276943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508474+0000","last_change":"2024-04-25T01:08:24.495211+0000","last_active":"2024-04-25T01:08:29.508474+0000","last_peered":"2024-04-25T01:08:29.508474+0000","last_clean":"2024-04-25T01:08:29.508474+0000","last_became_active":"2024-04-25T01:08:24.494930+0000","last_became_peered":"2024-04-25T01:08:24.494930+0000","last_unstale":"2024-04-25T01:08:29.508474+0000","last_undegraded":"2024-04-25T01:08:29.508474+0000","last_fullsized":"2024-04-25T01:08:29.508474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825747+0000","last_change":"2024-04-25T01:08:23.461624+0000","last_active":"2024-04-25T01:08:29.825747+0000","last_peered":"2024-04-25T01:08:29.825747+0000","last_clean":"2024-04-25T01:08:29.825747+0000","last_became_active":"2024-04-25T01:08:23.457067+0000","last_became_peered":"2024-04-25T01:08:23.457067+0000","last_unstale":"2024-04-25T01:08:29.825747+0000","last_undegraded":"2024-04-25T01:08:29.825747+0000","last_fullsized":"2024-04-25T01:08:29.825747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829168+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:08:29.829168+0000","last_peered":"2024-04-25T01:08:29.829168+0000","last_clean":"2024-04-25T01:08:29.829168+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:08:29.829168+0000","last_undegraded":"2024-04-25T01:08:29.829168+0000","last_fullsized":"2024-04-25T01:08:29.829168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826455+0000","last_change":"2024-04-25T01:08:28.861776+0000","last_active":"2024-04-25T01:08:29.826455+0000","last_peered":"2024-04-25T01:08:29.826455+0000","last_clean":"2024-04-25T01:08:29.826455+0000","last_became_active":"2024-04-25T01:08:28.860715+0000","last_became_peered":"2024-04-25T01:08:28.860715+0000","last_unstale":"2024-04-25T01:08:29.826455+0000","last_undegraded":"2024-04-25T01:08:29.826455+0000","last_fullsized":"2024-04-25T01:08:29.826455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826011+0000","last_change":"2024-04-25T01:08:24.523631+0000","last_active":"2024-04-25T01:08:29.826011+0000","last_peered":"2024-04-25T01:08:29.826011+0000","last_clean":"2024-04-25T01:08:29.826011+0000","last_became_active":"2024-04-25T01:08:24.523130+0000","last_became_peered":"2024-04-25T01:08:24.523130+0000","last_unstale":"2024-04-25T01:08:29.826011+0000","last_undegraded":"2024-04-25T01:08:29.826011+0000","last_fullsized":"2024-04-25T01:08:29.826011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:19.861291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826698+0000","last_change":"2024-04-25T01:08:23.470284+0000","last_active":"2024-04-25T01:08:29.826698+0000","last_peered":"2024-04-25T01:08:29.826698+0000","last_clean":"2024-04-25T01:08:29.826698+0000","last_became_active":"2024-04-25T01:08:23.470043+0000","last_became_peered":"2024-04-25T01:08:23.470043+0000","last_unstale":"2024-04-25T01:08:29.826698+0000","last_undegraded":"2024-04-25T01:08:29.826698+0000","last_fullsized":"2024-04-25T01:08:29.826698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827347+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827347+0000","last_peered":"2024-04-25T01:08:29.827347+0000","last_clean":"2024-04-25T01:08:29.827347+0000","last_became_active":"2024-04-25T01:08:28.858545+0000","last_became_peered":"2024-04-25T01:08:28.858545+0000","last_unstale":"2024-04-25T01:08:29.827347+0000","last_undegraded":"2024-04-25T01:08:29.827347+0000","last_fullsized":"2024-04-25T01:08:29.827347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826582+0000","last_change":"2024-04-25T01:08:24.490983+0000","last_active":"2024-04-25T01:08:29.826582+0000","last_peered":"2024-04-25T01:08:29.826582+0000","last_clean":"2024-04-25T01:08:29.826582+0000","last_became_active":"2024-04-25T01:08:24.490276+0000","last_became_peered":"2024-04-25T01:08:24.490276+0000","last_unstale":"2024-04-25T01:08:29.826582+0000","last_undegraded":"2024-04-25T01:08:29.826582+0000","last_fullsized":"2024-04-25T01:08:29.826582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:17.470136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829394+0000","last_change":"2024-04-25T01:08:23.459773+0000","last_active":"2024-04-25T01:08:29.829394+0000","last_peered":"2024-04-25T01:08:29.829394+0000","last_clean":"2024-04-25T01:08:29.829394+0000","last_became_active":"2024-04-25T01:08:23.459374+0000","last_became_peered":"2024-04-25T01:08:23.459374+0000","last_unstale":"2024-04-25T01:08:29.829394+0000","last_undegraded":"2024-04-25T01:08:29.829394+0000","last_fullsized":"2024-04-25T01:08:29.829394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827646+0000","last_change":"2024-04-25T01:08:28.542952+0000","last_active":"2024-04-25T01:08:29.827646+0000","last_peered":"2024-04-25T01:08:29.827646+0000","last_clean":"2024-04-25T01:08:29.827646+0000","last_became_active":"2024-04-25T01:08:28.542388+0000","last_became_peered":"2024-04-25T01:08:28.542388+0000","last_unstale":"2024-04-25T01:08:29.827646+0000","last_undegraded":"2024-04-25T01:08:29.827646+0000","last_fullsized":"2024-04-25T01:08:29.827646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:47.910183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828070+0000","last_change":"2024-04-25T01:08:24.525832+0000","last_active":"2024-04-25T01:08:29.828070+0000","last_peered":"2024-04-25T01:08:29.828070+0000","last_clean":"2024-04-25T01:08:29.828070+0000","last_became_active":"2024-04-25T01:08:24.524963+0000","last_became_peered":"2024-04-25T01:08:24.524963+0000","last_unstale":"2024-04-25T01:08:29.828070+0000","last_undegraded":"2024-04-25T01:08:29.828070+0000","last_fullsized":"2024-04-25T01:08:29.828070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:54.172408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828701+0000","last_change":"2024-04-25T01:08:23.448056+0000","last_active":"2024-04-25T01:08:29.828701+0000","last_peered":"2024-04-25T01:08:29.828701+0000","last_clean":"2024-04-25T01:08:29.828701+0000","last_became_active":"2024-04-25T01:08:23.447748+0000","last_became_peered":"2024-04-25T01:08:23.447748+0000","last_unstale":"2024-04-25T01:08:29.828701+0000","last_undegraded":"2024-04-25T01:08:29.828701+0000","last_fullsized":"2024-04-25T01:08:29.828701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824750+0000","last_change":"2024-04-25T01:08:28.860380+0000","last_active":"2024-04-25T01:08:29.824750+0000","last_peered":"2024-04-25T01:08:29.824750+0000","last_clean":"2024-04-25T01:08:29.824750+0000","last_became_active":"2024-04-25T01:08:28.859396+0000","last_became_peered":"2024-04-25T01:08:28.859396+0000","last_unstale":"2024-04-25T01:08:29.824750+0000","last_undegraded":"2024-04-25T01:08:29.824750+0000","last_fullsized":"2024-04-25T01:08:29.824750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:28.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826570+0000","last_change":"2024-04-25T01:08:24.524589+0000","last_active":"2024-04-25T01:08:29.826570+0000","last_peered":"2024-04-25T01:08:29.826570+0000","last_clean":"2024-04-25T01:08:29.826570+0000","last_became_active":"2024-04-25T01:08:24.523861+0000","last_became_peered":"2024-04-25T01:08:24.523861+0000","last_unstale":"2024-04-25T01:08:29.826570+0000","last_undegraded":"2024-04-25T01:08:29.826570+0000","last_fullsized":"2024-04-25T01:08:29.826570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:53.746720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512670+0000","last_change":"2024-04-25T01:08:23.445365+0000","last_active":"2024-04-25T01:08:29.512670+0000","last_peered":"2024-04-25T01:08:29.512670+0000","last_clean":"2024-04-25T01:08:29.512670+0000","last_became_active":"2024-04-25T01:08:23.444505+0000","last_became_peered":"2024-04-25T01:08:23.444505+0000","last_unstale":"2024-04-25T01:08:29.512670+0000","last_undegraded":"2024-04-25T01:08:29.512670+0000","last_fullsized":"2024-04-25T01:08:29.512670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:51.459603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510009+0000","last_change":"2024-04-25T01:08:28.549416+0000","last_active":"2024-04-25T01:08:29.510009+0000","last_peered":"2024-04-25T01:08:29.510009+0000","last_clean":"2024-04-25T01:08:29.510009+0000","last_became_active":"2024-04-25T01:08:28.547126+0000","last_became_peered":"2024-04-25T01:08:28.547126+0000","last_unstale":"2024-04-25T01:08:29.510009+0000","last_undegraded":"2024-04-25T01:08:29.510009+0000","last_fullsized":"2024-04-25T01:08:29.510009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:29.916469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828474+0000","last_change":"2024-04-25T01:08:24.514329+0000","last_active":"2024-04-25T01:08:29.828474+0000","last_peered":"2024-04-25T01:08:29.828474+0000","last_clean":"2024-04-25T01:08:29.828474+0000","last_became_active":"2024-04-25T01:08:24.513670+0000","last_became_peered":"2024-04-25T01:08:24.513670+0000","last_unstale":"2024-04-25T01:08:29.828474+0000","last_undegraded":"2024-04-25T01:08:29.828474+0000","last_fullsized":"2024-04-25T01:08:29.828474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.823985+0000","last_change":"2024-04-25T01:08:23.449156+0000","last_active":"2024-04-25T01:08:29.823985+0000","last_peered":"2024-04-25T01:08:29.823985+0000","last_clean":"2024-04-25T01:08:29.823985+0000","last_became_active":"2024-04-25T01:08:23.448724+0000","last_became_peered":"2024-04-25T01:08:23.448724+0000","last_unstale":"2024-04-25T01:08:29.823985+0000","last_undegraded":"2024-04-25T01:08:29.823985+0000","last_fullsized":"2024-04-25T01:08:29.823985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829295+0000","last_change":"2024-04-25T01:08:28.538540+0000","last_active":"2024-04-25T01:08:29.829295+0000","last_peered":"2024-04-25T01:08:29.829295+0000","last_clean":"2024-04-25T01:08:29.829295+0000","last_became_active":"2024-04-25T01:08:28.537662+0000","last_became_peered":"2024-04-25T01:08:28.537662+0000","last_unstale":"2024-04-25T01:08:29.829295+0000","last_undegraded":"2024-04-25T01:08:29.829295+0000","last_fullsized":"2024-04-25T01:08:29.829295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:06.032667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824689+0000","last_change":"2024-04-25T01:08:24.499087+0000","last_active":"2024-04-25T01:08:29.824689+0000","last_peered":"2024-04-25T01:08:29.824689+0000","last_clean":"2024-04-25T01:08:29.824689+0000","last_became_active":"2024-04-25T01:08:24.498181+0000","last_became_peered":"2024-04-25T01:08:24.498181+0000","last_unstale":"2024-04-25T01:08:29.824689+0000","last_undegraded":"2024-04-25T01:08:29.824689+0000","last_fullsized":"2024-04-25T01:08:29.824689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825380+0000","last_change":"2024-04-25T01:08:23.466687+0000","last_active":"2024-04-25T01:08:29.825380+0000","last_peered":"2024-04-25T01:08:29.825380+0000","last_clean":"2024-04-25T01:08:29.825380+0000","last_became_active":"2024-04-25T01:08:23.465922+0000","last_became_peered":"2024-04-25T01:08:23.465922+0000","last_unstale":"2024-04-25T01:08:29.825380+0000","last_undegraded":"2024-04-25T01:08:29.825380+0000","last_fullsized":"2024-04-25T01:08:29.825380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510286+0000","last_change":"2024-04-25T01:08:28.861374+0000","last_active":"2024-04-25T01:08:29.510286+0000","last_peered":"2024-04-25T01:08:29.510286+0000","last_clean":"2024-04-25T01:08:29.510286+0000","last_became_active":"2024-04-25T01:08:28.857471+0000","last_became_peered":"2024-04-25T01:08:28.857471+0000","last_unstale":"2024-04-25T01:08:29.510286+0000","last_undegraded":"2024-04-25T01:08:29.510286+0000","last_fullsized":"2024-04-25T01:08:29.510286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:50.580633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827859+0000","last_change":"2024-04-25T01:08:24.524773+0000","last_active":"2024-04-25T01:08:29.827859+0000","last_peered":"2024-04-25T01:08:29.827859+0000","last_clean":"2024-04-25T01:08:29.827859+0000","last_became_active":"2024-04-25T01:08:24.523273+0000","last_became_peered":"2024-04-25T01:08:24.523273+0000","last_unstale":"2024-04-25T01:08:29.827859+0000","last_undegraded":"2024-04-25T01:08:29.827859+0000","last_fullsized":"2024-04-25T01:08:29.827859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:53.559257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826090+0000","last_change":"2024-04-25T01:08:23.478405+0000","last_active":"2024-04-25T01:08:29.826090+0000","last_peered":"2024-04-25T01:08:29.826090+0000","last_clean":"2024-04-25T01:08:29.826090+0000","last_became_active":"2024-04-25T01:08:23.477864+0000","last_became_peered":"2024-04-25T01:08:23.477864+0000","last_unstale":"2024-04-25T01:08:29.826090+0000","last_undegraded":"2024-04-25T01:08:29.826090+0000","last_fullsized":"2024-04-25T01:08:29.826090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:14.206151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509288+0000","last_change":"2024-04-25T01:08:28.859485+0000","last_active":"2024-04-25T01:08:29.509288+0000","last_peered":"2024-04-25T01:08:29.509288+0000","last_clean":"2024-04-25T01:08:29.509288+0000","last_became_active":"2024-04-25T01:08:28.858350+0000","last_became_peered":"2024-04-25T01:08:28.858350+0000","last_unstale":"2024-04-25T01:08:29.509288+0000","last_undegraded":"2024-04-25T01:08:29.509288+0000","last_fullsized":"2024-04-25T01:08:29.509288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:27.377063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:24.502358+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:24.502144+0000","last_became_peered":"2024-04-25T01:08:24.502144+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:17.614542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827872+0000","last_change":"2024-04-25T01:08:23.444558+0000","last_active":"2024-04-25T01:08:29.827872+0000","last_peered":"2024-04-25T01:08:29.827872+0000","last_clean":"2024-04-25T01:08:29.827872+0000","last_became_active":"2024-04-25T01:08:23.444246+0000","last_became_peered":"2024-04-25T01:08:23.444246+0000","last_unstale":"2024-04-25T01:08:29.827872+0000","last_undegraded":"2024-04-25T01:08:29.827872+0000","last_fullsized":"2024-04-25T01:08:29.827872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827499+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827499+0000","last_peered":"2024-04-25T01:08:29.827499+0000","last_clean":"2024-04-25T01:08:29.827499+0000","last_became_active":"2024-04-25T01:08:28.858324+0000","last_became_peered":"2024-04-25T01:08:28.858324+0000","last_unstale":"2024-04-25T01:08:29.827499+0000","last_undegraded":"2024-04-25T01:08:29.827499+0000","last_fullsized":"2024-04-25T01:08:29.827499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:49.074275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825751+0000","last_change":"2024-04-25T01:08:24.520516+0000","last_active":"2024-04-25T01:08:29.825751+0000","last_peered":"2024-04-25T01:08:29.825751+0000","last_clean":"2024-04-25T01:08:29.825751+0000","last_became_active":"2024-04-25T01:08:24.519967+0000","last_became_peered":"2024-04-25T01:08:24.519967+0000","last_unstale":"2024-04-25T01:08:29.825751+0000","last_undegraded":"2024-04-25T01:08:29.825751+0000","last_fullsized":"2024-04-25T01:08:29.825751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:50.176772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:33.334782+0000","last_change":"2024-04-25T01:08:23.460787+0000","last_active":"2024-04-25T01:08:33.334782+0000","last_peered":"2024-04-25T01:08:33.334782+0000","last_clean":"2024-04-25T01:08:33.334782+0000","last_became_active":"2024-04-25T01:08:23.455085+0000","last_became_peered":"2024-04-25T01:08:23.455085+0000","last_unstale":"2024-04-25T01:08:33.334782+0000","last_undegraded":"2024-04-25T01:08:33.334782+0000","last_fullsized":"2024-04-25T01:08:33.334782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825396+0000","last_change":"2024-04-25T01:08:28.536251+0000","last_active":"2024-04-25T01:08:29.825396+0000","last_peered":"2024-04-25T01:08:29.825396+0000","last_clean":"2024-04-25T01:08:29.825396+0000","last_became_active":"2024-04-25T01:08:28.535359+0000","last_became_peered":"2024-04-25T01:08:28.535359+0000","last_unstale":"2024-04-25T01:08:29.825396+0000","last_undegraded":"2024-04-25T01:08:29.825396+0000","last_fullsized":"2024-04-25T01:08:29.825396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511615+0000","last_change":"2024-04-25T01:08:24.505163+0000","last_active":"2024-04-25T01:08:29.511615+0000","last_peered":"2024-04-25T01:08:29.511615+0000","last_clean":"2024-04-25T01:08:29.511615+0000","last_became_active":"2024-04-25T01:08:24.504390+0000","last_became_peered":"2024-04-25T01:08:24.504390+0000","last_unstale":"2024-04-25T01:08:29.511615+0000","last_undegraded":"2024-04-25T01:08:29.511615+0000","last_fullsized":"2024-04-25T01:08:29.511615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829299+0000","last_change":"2024-04-25T01:08:23.484553+0000","last_active":"2024-04-25T01:08:29.829299+0000","last_peered":"2024-04-25T01:08:29.829299+0000","last_clean":"2024-04-25T01:08:29.829299+0000","last_became_active":"2024-04-25T01:08:23.484289+0000","last_became_peered":"2024-04-25T01:08:23.484289+0000","last_unstale":"2024-04-25T01:08:29.829299+0000","last_undegraded":"2024-04-25T01:08:29.829299+0000","last_fullsized":"2024-04-25T01:08:29.829299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511206+0000","last_change":"2024-04-25T01:08:28.538547+0000","last_active":"2024-04-25T01:08:29.511206+0000","last_peered":"2024-04-25T01:08:29.511206+0000","last_clean":"2024-04-25T01:08:29.511206+0000","last_became_active":"2024-04-25T01:08:28.537497+0000","last_became_peered":"2024-04-25T01:08:28.537497+0000","last_unstale":"2024-04-25T01:08:29.511206+0000","last_undegraded":"2024-04-25T01:08:29.511206+0000","last_fullsized":"2024-04-25T01:08:29.511206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830135+0000","last_change":"2024-04-25T01:08:24.493527+0000","last_active":"2024-04-25T01:08:29.830135+0000","last_peered":"2024-04-25T01:08:29.830135+0000","last_clean":"2024-04-25T01:08:29.830135+0000","last_became_active":"2024-04-25T01:08:24.493206+0000","last_became_peered":"2024-04-25T01:08:24.493206+0000","last_unstale":"2024-04-25T01:08:29.830135+0000","last_undegraded":"2024-04-25T01:08:29.830135+0000","last_fullsized":"2024-04-25T01:08:29.830135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826688+0000","last_change":"2024-04-25T01:08:23.476900+0000","last_active":"2024-04-25T01:08:29.826688+0000","last_peered":"2024-04-25T01:08:29.826688+0000","last_clean":"2024-04-25T01:08:29.826688+0000","last_became_active":"2024-04-25T01:08:23.474893+0000","last_became_peered":"2024-04-25T01:08:23.474893+0000","last_unstale":"2024-04-25T01:08:29.826688+0000","last_undegraded":"2024-04-25T01:08:29.826688+0000","last_fullsized":"2024-04-25T01:08:29.826688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:25.208026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508202+0000","last_change":"2024-04-25T01:08:28.545577+0000","last_active":"2024-04-25T01:08:29.508202+0000","last_peered":"2024-04-25T01:08:29.508202+0000","last_clean":"2024-04-25T01:08:29.508202+0000","last_became_active":"2024-04-25T01:08:28.540902+0000","last_became_peered":"2024-04-25T01:08:28.540902+0000","last_unstale":"2024-04-25T01:08:29.508202+0000","last_undegraded":"2024-04-25T01:08:29.508202+0000","last_fullsized":"2024-04-25T01:08:29.508202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:14.257846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826495+0000","last_change":"2024-04-25T01:08:24.515829+0000","last_active":"2024-04-25T01:08:29.826495+0000","last_peered":"2024-04-25T01:08:29.826495+0000","last_clean":"2024-04-25T01:08:29.826495+0000","last_became_active":"2024-04-25T01:08:24.515267+0000","last_became_peered":"2024-04-25T01:08:24.515267+0000","last_unstale":"2024-04-25T01:08:29.826495+0000","last_undegraded":"2024-04-25T01:08:29.826495+0000","last_fullsized":"2024-04-25T01:08:29.826495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:51.870029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829322+0000","last_change":"2024-04-25T01:08:23.478857+0000","last_active":"2024-04-25T01:08:29.829322+0000","last_peered":"2024-04-25T01:08:29.829322+0000","last_clean":"2024-04-25T01:08:29.829322+0000","last_became_active":"2024-04-25T01:08:23.478215+0000","last_became_peered":"2024-04-25T01:08:23.478215+0000","last_unstale":"2024-04-25T01:08:29.829322+0000","last_undegraded":"2024-04-25T01:08:29.829322+0000","last_fullsized":"2024-04-25T01:08:29.829322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508255+0000","last_change":"2024-04-25T01:08:28.863219+0000","last_active":"2024-04-25T01:08:29.508255+0000","last_peered":"2024-04-25T01:08:29.508255+0000","last_clean":"2024-04-25T01:08:29.508255+0000","last_became_active":"2024-04-25T01:08:28.862202+0000","last_became_peered":"2024-04-25T01:08:28.862202+0000","last_unstale":"2024-04-25T01:08:29.508255+0000","last_undegraded":"2024-04-25T01:08:29.508255+0000","last_fullsized":"2024-04-25T01:08:29.508255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:58.057652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829042+0000","last_change":"2024-04-25T01:08:28.864029+0000","last_active":"2024-04-25T01:08:29.829042+0000","last_peered":"2024-04-25T01:08:29.829042+0000","last_clean":"2024-04-25T01:08:29.829042+0000","last_became_active":"2024-04-25T01:08:28.863040+0000","last_became_peered":"2024-04-25T01:08:28.863040+0000","last_unstale":"2024-04-25T01:08:29.829042+0000","last_undegraded":"2024-04-25T01:08:29.829042+0000","last_fullsized":"2024-04-25T01:08:29.829042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:55.932667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827904+0000","last_change":"2024-04-25T01:08:24.472094+0000","last_active":"2024-04-25T01:08:29.827904+0000","last_peered":"2024-04-25T01:08:29.827904+0000","last_clean":"2024-04-25T01:08:29.827904+0000","last_became_active":"2024-04-25T01:08:24.471381+0000","last_became_peered":"2024-04-25T01:08:24.471381+0000","last_unstale":"2024-04-25T01:08:29.827904+0000","last_undegraded":"2024-04-25T01:08:29.827904+0000","last_fullsized":"2024-04-25T01:08:29.827904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825540+0000","last_change":"2024-04-25T01:08:23.470504+0000","last_active":"2024-04-25T01:08:29.825540+0000","last_peered":"2024-04-25T01:08:29.825540+0000","last_clean":"2024-04-25T01:08:29.825540+0000","last_became_active":"2024-04-25T01:08:23.470060+0000","last_became_peered":"2024-04-25T01:08:23.470060+0000","last_unstale":"2024-04-25T01:08:29.825540+0000","last_undegraded":"2024-04-25T01:08:29.825540+0000","last_fullsized":"2024-04-25T01:08:29.825540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:52.050718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829192+0000","last_change":"2024-04-25T01:08:28.544590+0000","last_active":"2024-04-25T01:08:29.829192+0000","last_peered":"2024-04-25T01:08:29.829192+0000","last_clean":"2024-04-25T01:08:29.829192+0000","last_became_active":"2024-04-25T01:08:28.543195+0000","last_became_peered":"2024-04-25T01:08:28.543195+0000","last_unstale":"2024-04-25T01:08:29.829192+0000","last_undegraded":"2024-04-25T01:08:29.829192+0000","last_fullsized":"2024-04-25T01:08:29.829192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508183+0000","last_change":"2024-04-25T01:08:24.494258+0000","last_active":"2024-04-25T01:08:29.508183+0000","last_peered":"2024-04-25T01:08:29.508183+0000","last_clean":"2024-04-25T01:08:29.508183+0000","last_became_active":"2024-04-25T01:08:24.493935+0000","last_became_peered":"2024-04-25T01:08:24.493935+0000","last_unstale":"2024-04-25T01:08:29.508183+0000","last_undegraded":"2024-04-25T01:08:29.508183+0000","last_fullsized":"2024-04-25T01:08:29.508183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508536+0000","last_change":"2024-04-25T01:08:23.463618+0000","last_active":"2024-04-25T01:08:29.508536+0000","last_peered":"2024-04-25T01:08:29.508536+0000","last_clean":"2024-04-25T01:08:29.508536+0000","last_became_active":"2024-04-25T01:08:23.463303+0000","last_became_peered":"2024-04-25T01:08:23.463303+0000","last_unstale":"2024-04-25T01:08:29.508536+0000","last_undegraded":"2024-04-25T01:08:29.508536+0000","last_fullsized":"2024-04-25T01:08:29.508536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828071+0000","last_change":"2024-04-25T01:08:23.465959+0000","last_active":"2024-04-25T01:08:29.828071+0000","last_peered":"2024-04-25T01:08:29.828071+0000","last_clean":"2024-04-25T01:08:29.828071+0000","last_became_active":"2024-04-25T01:08:23.465518+0000","last_became_peered":"2024-04-25T01:08:23.465518+0000","last_unstale":"2024-04-25T01:08:29.828071+0000","last_undegraded":"2024-04-25T01:08:29.828071+0000","last_fullsized":"2024-04-25T01:08:29.828071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827782+0000","last_change":"2024-04-25T01:08:24.493189+0000","last_active":"2024-04-25T01:08:29.827782+0000","last_peered":"2024-04-25T01:08:29.827782+0000","last_clean":"2024-04-25T01:08:29.827782+0000","last_became_active":"2024-04-25T01:08:24.473740+0000","last_became_peered":"2024-04-25T01:08:24.473740+0000","last_unstale":"2024-04-25T01:08:29.827782+0000","last_undegraded":"2024-04-25T01:08:29.827782+0000","last_fullsized":"2024-04-25T01:08:29.827782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825925+0000","last_change":"2024-04-25T01:08:28.542530+0000","last_active":"2024-04-25T01:08:29.825925+0000","last_peered":"2024-04-25T01:08:29.825925+0000","last_clean":"2024-04-25T01:08:29.825925+0000","last_became_active":"2024-04-25T01:08:28.541445+0000","last_became_peered":"2024-04-25T01:08:28.541445+0000","last_unstale":"2024-04-25T01:08:29.825925+0000","last_undegraded":"2024-04-25T01:08:29.825925+0000","last_fullsized":"2024-04-25T01:08:29.825925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:21.963089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508697+0000","last_change":"2024-04-25T01:08:23.475915+0000","last_active":"2024-04-25T01:08:29.508697+0000","last_peered":"2024-04-25T01:08:29.508697+0000","last_clean":"2024-04-25T01:08:29.508697+0000","last_became_active":"2024-04-25T01:08:23.475628+0000","last_became_peered":"2024-04-25T01:08:23.475628+0000","last_unstale":"2024-04-25T01:08:29.508697+0000","last_undegraded":"2024-04-25T01:08:29.508697+0000","last_fullsized":"2024-04-25T01:08:29.508697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511101+0000","last_change":"2024-04-25T01:08:24.476673+0000","last_active":"2024-04-25T01:08:29.511101+0000","last_peered":"2024-04-25T01:08:29.511101+0000","last_clean":"2024-04-25T01:08:29.511101+0000","last_became_active":"2024-04-25T01:08:24.471388+0000","last_became_peered":"2024-04-25T01:08:24.471388+0000","last_unstale":"2024-04-25T01:08:29.511101+0000","last_undegraded":"2024-04-25T01:08:29.511101+0000","last_fullsized":"2024-04-25T01:08:29.511101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:36.539667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509254+0000","last_change":"2024-04-25T01:08:28.539720+0000","last_active":"2024-04-25T01:08:29.509254+0000","last_peered":"2024-04-25T01:08:29.509254+0000","last_clean":"2024-04-25T01:08:29.509254+0000","last_became_active":"2024-04-25T01:08:28.538885+0000","last_became_peered":"2024-04-25T01:08:28.538885+0000","last_unstale":"2024-04-25T01:08:29.509254+0000","last_undegraded":"2024-04-25T01:08:29.509254+0000","last_fullsized":"2024-04-25T01:08:29.509254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:51.273909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824379+0000","last_change":"2024-04-25T01:08:24.499102+0000","last_active":"2024-04-25T01:08:29.824379+0000","last_peered":"2024-04-25T01:08:29.824379+0000","last_clean":"2024-04-25T01:08:29.824379+0000","last_became_active":"2024-04-25T01:08:24.498282+0000","last_became_peered":"2024-04-25T01:08:24.498282+0000","last_unstale":"2024-04-25T01:08:29.824379+0000","last_undegraded":"2024-04-25T01:08:29.824379+0000","last_fullsized":"2024-04-25T01:08:29.824379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824785+0000","last_change":"2024-04-25T01:08:24.477024+0000","last_active":"2024-04-25T01:08:29.824785+0000","last_peered":"2024-04-25T01:08:29.824785+0000","last_clean":"2024-04-25T01:08:29.824785+0000","last_became_active":"2024-04-25T01:08:24.476723+0000","last_became_peered":"2024-04-25T01:08:24.476723+0000","last_unstale":"2024-04-25T01:08:29.824785+0000","last_undegraded":"2024-04-25T01:08:29.824785+0000","last_fullsized":"2024-04-25T01:08:29.824785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825769+0000","last_change":"2024-04-25T01:08:24.492781+0000","last_active":"2024-04-25T01:08:29.825769+0000","last_peered":"2024-04-25T01:08:29.825769+0000","last_clean":"2024-04-25T01:08:29.825769+0000","last_became_active":"2024-04-25T01:08:24.492041+0000","last_became_peered":"2024-04-25T01:08:24.492041+0000","last_unstale":"2024-04-25T01:08:29.825769+0000","last_undegraded":"2024-04-25T01:08:29.825769+0000","last_fullsized":"2024-04-25T01:08:29.825769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:35.171404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827611+0000","last_change":"2024-04-25T01:08:24.493915+0000","last_active":"2024-04-25T01:08:29.827611+0000","last_peered":"2024-04-25T01:08:29.827611+0000","last_clean":"2024-04-25T01:08:29.827611+0000","last_became_active":"2024-04-25T01:08:24.493620+0000","last_became_peered":"2024-04-25T01:08:24.493620+0000","last_unstale":"2024-04-25T01:08:29.827611+0000","last_undegraded":"2024-04-25T01:08:29.827611+0000","last_fullsized":"2024-04-25T01:08:29.827611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:42.788724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509565+0000","last_change":"2024-04-25T01:08:24.491720+0000","last_active":"2024-04-25T01:08:29.509565+0000","last_peered":"2024-04-25T01:08:29.509565+0000","last_clean":"2024-04-25T01:08:29.509565+0000","last_became_active":"2024-04-25T01:08:24.491474+0000","last_became_peered":"2024-04-25T01:08:24.491474+0000","last_unstale":"2024-04-25T01:08:29.509565+0000","last_undegraded":"2024-04-25T01:08:29.509565+0000","last_fullsized":"2024-04-25T01:08:29.509565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828668+0000","last_change":"2024-04-25T01:08:24.472108+0000","last_active":"2024-04-25T01:08:29.828668+0000","last_peered":"2024-04-25T01:08:29.828668+0000","last_clean":"2024-04-25T01:08:29.828668+0000","last_became_active":"2024-04-25T01:08:24.471688+0000","last_became_peered":"2024-04-25T01:08:24.471688+0000","last_unstale":"2024-04-25T01:08:29.828668+0000","last_undegraded":"2024-04-25T01:08:29.828668+0000","last_fullsized":"2024-04-25T01:08:29.828668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829144+0000","last_change":"2024-04-25T01:08:24.515413+0000","last_active":"2024-04-25T01:08:29.829144+0000","last_peered":"2024-04-25T01:08:29.829144+0000","last_clean":"2024-04-25T01:08:29.829144+0000","last_became_active":"2024-04-25T01:08:24.514573+0000","last_became_peered":"2024-04-25T01:08:24.514573+0000","last_unstale":"2024-04-25T01:08:29.829144+0000","last_undegraded":"2024-04-25T01:08:29.829144+0000","last_fullsized":"2024-04-25T01:08:29.829144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510173+0000","last_change":"2024-04-25T01:08:24.487417+0000","last_active":"2024-04-25T01:08:29.510173+0000","last_peered":"2024-04-25T01:08:29.510173+0000","last_clean":"2024-04-25T01:08:29.510173+0000","last_became_active":"2024-04-25T01:08:24.487135+0000","last_became_peered":"2024-04-25T01:08:24.487135+0000","last_unstale":"2024-04-25T01:08:29.510173+0000","last_undegraded":"2024-04-25T01:08:29.510173+0000","last_fullsized":"2024-04-25T01:08:29.510173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825289+0000","last_change":"2024-04-25T01:08:24.484688+0000","last_active":"2024-04-25T01:08:29.825289+0000","last_peered":"2024-04-25T01:08:29.825289+0000","last_clean":"2024-04-25T01:08:29.825289+0000","last_became_active":"2024-04-25T01:08:24.483926+0000","last_became_peered":"2024-04-25T01:08:24.483926+0000","last_unstale":"2024-04-25T01:08:29.825289+0000","last_undegraded":"2024-04-25T01:08:29.825289+0000","last_fullsized":"2024-04-25T01:08:29.825289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508350+0000","last_change":"2024-04-25T01:08:24.487479+0000","last_active":"2024-04-25T01:08:29.508350+0000","last_peered":"2024-04-25T01:08:29.508350+0000","last_clean":"2024-04-25T01:08:29.508350+0000","last_became_active":"2024-04-25T01:08:24.487177+0000","last_became_peered":"2024-04-25T01:08:24.487177+0000","last_unstale":"2024-04-25T01:08:29.508350+0000","last_undegraded":"2024-04-25T01:08:29.508350+0000","last_fullsized":"2024-04-25T01:08:29.508350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825549+0000","last_change":"2024-04-25T01:08:24.516077+0000","last_active":"2024-04-25T01:08:29.825549+0000","last_peered":"2024-04-25T01:08:29.825549+0000","last_clean":"2024-04-25T01:08:29.825549+0000","last_became_active":"2024-04-25T01:08:24.515560+0000","last_became_peered":"2024-04-25T01:08:24.515560+0000","last_unstale":"2024-04-25T01:08:29.825549+0000","last_undegraded":"2024-04-25T01:08:29.825549+0000","last_fullsized":"2024-04-25T01:08:29.825549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:30.731050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827066+0000","last_change":"2024-04-25T01:08:24.497188+0000","last_active":"2024-04-25T01:08:29.827066+0000","last_peered":"2024-04-25T01:08:29.827066+0000","last_clean":"2024-04-25T01:08:29.827066+0000","last_became_active":"2024-04-25T01:08:24.494236+0000","last_became_peered":"2024-04-25T01:08:24.494236+0000","last_unstale":"2024-04-25T01:08:29.827066+0000","last_undegraded":"2024-04-25T01:08:29.827066+0000","last_fullsized":"2024-04-25T01:08:29.827066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:27.907903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825891+0000","last_change":"2024-04-25T01:08:24.497623+0000","last_active":"2024-04-25T01:08:29.825891+0000","last_peered":"2024-04-25T01:08:29.825891+0000","last_clean":"2024-04-25T01:08:29.825891+0000","last_became_active":"2024-04-25T01:08:24.488300+0000","last_became_peered":"2024-04-25T01:08:24.488300+0000","last_unstale":"2024-04-25T01:08:29.825891+0000","last_undegraded":"2024-04-25T01:08:29.825891+0000","last_fullsized":"2024-04-25T01:08:29.825891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826575+0000","last_change":"2024-04-25T01:08:24.525066+0000","last_active":"2024-04-25T01:08:29.826575+0000","last_peered":"2024-04-25T01:08:29.826575+0000","last_clean":"2024-04-25T01:08:29.826575+0000","last_became_active":"2024-04-25T01:08:24.524022+0000","last_became_peered":"2024-04-25T01:08:24.524022+0000","last_unstale":"2024-04-25T01:08:29.826575+0000","last_undegraded":"2024-04-25T01:08:29.826575+0000","last_fullsized":"2024-04-25T01:08:29.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:33.317294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828523+0000","last_change":"2024-04-25T01:08:24.474988+0000","last_active":"2024-04-25T01:08:29.828523+0000","last_peered":"2024-04-25T01:08:29.828523+0000","last_clean":"2024-04-25T01:08:29.828523+0000","last_became_active":"2024-04-25T01:08:24.472840+0000","last_became_peered":"2024-04-25T01:08:24.472840+0000","last_unstale":"2024-04-25T01:08:29.828523+0000","last_undegraded":"2024-04-25T01:08:29.828523+0000","last_fullsized":"2024-04-25T01:08:29.828523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827354+0000","last_change":"2024-04-25T01:08:24.504414+0000","last_active":"2024-04-25T01:08:29.827354+0000","last_peered":"2024-04-25T01:08:29.827354+0000","last_clean":"2024-04-25T01:08:29.827354+0000","last_became_active":"2024-04-25T01:08:24.503622+0000","last_became_peered":"2024-04-25T01:08:24.503622+0000","last_unstale":"2024-04-25T01:08:29.827354+0000","last_undegraded":"2024-04-25T01:08:29.827354+0000","last_fullsized":"2024-04-25T01:08:29.827354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:01.970030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825048+0000","last_change":"2024-04-25T01:08:24.496532+0000","last_active":"2024-04-25T01:08:29.825048+0000","last_peered":"2024-04-25T01:08:29.825048+0000","last_clean":"2024-04-25T01:08:29.825048+0000","last_became_active":"2024-04-25T01:08:24.495812+0000","last_became_peered":"2024-04-25T01:08:24.495812+0000","last_unstale":"2024-04-25T01:08:29.825048+0000","last_undegraded":"2024-04-25T01:08:29.825048+0000","last_fullsized":"2024-04-25T01:08:29.825048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828165+0000","last_change":"2024-04-25T01:08:24.461285+0000","last_active":"2024-04-25T01:08:29.828165+0000","last_peered":"2024-04-25T01:08:29.828165+0000","last_clean":"2024-04-25T01:08:29.828165+0000","last_became_active":"2024-04-25T01:08:24.460929+0000","last_became_peered":"2024-04-25T01:08:24.460929+0000","last_unstale":"2024-04-25T01:08:29.828165+0000","last_undegraded":"2024-04-25T01:08:29.828165+0000","last_fullsized":"2024-04-25T01:08:29.828165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827126+0000","last_change":"2024-04-25T01:08:24.493618+0000","last_active":"2024-04-25T01:08:29.827126+0000","last_peered":"2024-04-25T01:08:29.827126+0000","last_clean":"2024-04-25T01:08:29.827126+0000","last_became_active":"2024-04-25T01:08:24.489439+0000","last_became_peered":"2024-04-25T01:08:24.489439+0000","last_unstale":"2024-04-25T01:08:29.827126+0000","last_undegraded":"2024-04-25T01:08:29.827126+0000","last_fullsized":"2024-04-25T01:08:29.827126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:43.940192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508267+0000","last_change":"2024-04-25T01:08:24.478846+0000","last_active":"2024-04-25T01:08:29.508267+0000","last_peered":"2024-04-25T01:08:29.508267+0000","last_clean":"2024-04-25T01:08:29.508267+0000","last_became_active":"2024-04-25T01:08:24.478580+0000","last_became_peered":"2024-04-25T01:08:24.478580+0000","last_unstale":"2024-04-25T01:08:29.508267+0000","last_undegraded":"2024-04-25T01:08:29.508267+0000","last_fullsized":"2024-04-25T01:08:29.508267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511438+0000","last_change":"2024-04-25T01:08:24.481980+0000","last_active":"2024-04-25T01:08:29.511438+0000","last_peered":"2024-04-25T01:08:29.511438+0000","last_clean":"2024-04-25T01:08:29.511438+0000","last_became_active":"2024-04-25T01:08:24.480214+0000","last_became_peered":"2024-04-25T01:08:24.480214+0000","last_unstale":"2024-04-25T01:08:29.511438+0000","last_undegraded":"2024-04-25T01:08:29.511438+0000","last_fullsized":"2024-04-25T01:08:29.511438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826879+0000","last_change":"2024-04-25T01:08:24.480212+0000","last_active":"2024-04-25T01:08:29.826879+0000","last_peered":"2024-04-25T01:08:29.826879+0000","last_clean":"2024-04-25T01:08:29.826879+0000","last_became_active":"2024-04-25T01:08:24.477781+0000","last_became_peered":"2024-04-25T01:08:24.477781+0000","last_unstale":"2024-04-25T01:08:29.826879+0000","last_undegraded":"2024-04-25T01:08:29.826879+0000","last_fullsized":"2024-04-25T01:08:29.826879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829093+0000","last_change":"2024-04-25T01:08:24.480926+0000","last_active":"2024-04-25T01:08:29.829093+0000","last_peered":"2024-04-25T01:08:29.829093+0000","last_clean":"2024-04-25T01:08:29.829093+0000","last_became_active":"2024-04-25T01:08:24.477787+0000","last_became_peered":"2024-04-25T01:08:24.477787+0000","last_unstale":"2024-04-25T01:08:29.829093+0000","last_undegraded":"2024-04-25T01:08:29.829093+0000","last_fullsized":"2024-04-25T01:08:29.829093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827821+0000","last_change":"2024-04-25T01:08:24.525497+0000","last_active":"2024-04-25T01:08:29.827821+0000","last_peered":"2024-04-25T01:08:29.827821+0000","last_clean":"2024-04-25T01:08:29.827821+0000","last_became_active":"2024-04-25T01:08:24.524512+0000","last_became_peered":"2024-04-25T01:08:24.524512+0000","last_unstale":"2024-04-25T01:08:29.827821+0000","last_undegraded":"2024-04-25T01:08:29.827821+0000","last_fullsized":"2024-04-25T01:08:29.827821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:35.642716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507922+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.507922+0000","last_peered":"2024-04-25T01:08:29.507922+0000","last_clean":"2024-04-25T01:08:29.507922+0000","last_became_active":"2024-04-25T01:08:24.486493+0000","last_became_peered":"2024-04-25T01:08:24.486493+0000","last_unstale":"2024-04-25T01:08:29.507922+0000","last_undegraded":"2024-04-25T01:08:29.507922+0000","last_fullsized":"2024-04-25T01:08:29.507922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828202+0000","last_change":"2024-04-25T01:08:24.510776+0000","last_active":"2024-04-25T01:08:29.828202+0000","last_peered":"2024-04-25T01:08:29.828202+0000","last_clean":"2024-04-25T01:08:29.828202+0000","last_became_active":"2024-04-25T01:08:24.509980+0000","last_became_peered":"2024-04-25T01:08:24.509980+0000","last_unstale":"2024-04-25T01:08:29.828202+0000","last_undegraded":"2024-04-25T01:08:29.828202+0000","last_fullsized":"2024-04-25T01:08:29.828202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825321+0000","last_change":"2024-04-25T01:08:24.497109+0000","last_active":"2024-04-25T01:08:29.825321+0000","last_peered":"2024-04-25T01:08:29.825321+0000","last_clean":"2024-04-25T01:08:29.825321+0000","last_became_active":"2024-04-25T01:08:24.496213+0000","last_became_peered":"2024-04-25T01:08:24.496213+0000","last_unstale":"2024-04-25T01:08:29.825321+0000","last_undegraded":"2024-04-25T01:08:29.825321+0000","last_fullsized":"2024-04-25T01:08:29.825321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827021+0000","last_change":"2024-04-25T01:08:24.524338+0000","last_active":"2024-04-25T01:08:29.827021+0000","last_peered":"2024-04-25T01:08:29.827021+0000","last_clean":"2024-04-25T01:08:29.827021+0000","last_became_active":"2024-04-25T01:08:24.523936+0000","last_became_peered":"2024-04-25T01:08:24.523936+0000","last_unstale":"2024-04-25T01:08:29.827021+0000","last_undegraded":"2024-04-25T01:08:29.827021+0000","last_fullsized":"2024-04-25T01:08:29.827021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:50.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826302+0000","last_change":"2024-04-25T01:08:24.491543+0000","last_active":"2024-04-25T01:08:29.826302+0000","last_peered":"2024-04-25T01:08:29.826302+0000","last_clean":"2024-04-25T01:08:29.826302+0000","last_became_active":"2024-04-25T01:08:24.490866+0000","last_became_peered":"2024-04-25T01:08:24.490866+0000","last_unstale":"2024-04-25T01:08:29.826302+0000","last_undegraded":"2024-04-25T01:08:29.826302+0000","last_fullsized":"2024-04-25T01:08:29.826302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:38.093629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828785+0000","last_change":"2024-04-25T01:08:24.459335+0000","last_active":"2024-04-25T01:08:29.828785+0000","last_peered":"2024-04-25T01:08:29.828785+0000","last_clean":"2024-04-25T01:08:29.828785+0000","last_became_active":"2024-04-25T01:08:24.458795+0000","last_became_peered":"2024-04-25T01:08:24.458795+0000","last_unstale":"2024-04-25T01:08:29.828785+0000","last_undegraded":"2024-04-25T01:08:29.828785+0000","last_fullsized":"2024-04-25T01:08:29.828785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:10.682483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829652+0000","last_change":"2024-04-25T01:08:24.465465+0000","last_active":"2024-04-25T01:08:29.829652+0000","last_peered":"2024-04-25T01:08:29.829652+0000","last_clean":"2024-04-25T01:08:29.829652+0000","last_became_active":"2024-04-25T01:08:24.464433+0000","last_became_peered":"2024-04-25T01:08:24.464433+0000","last_unstale":"2024-04-25T01:08:29.829652+0000","last_undegraded":"2024-04-25T01:08:29.829652+0000","last_fullsized":"2024-04-25T01:08:29.829652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827333+0000","last_change":"2024-04-25T01:08:24.525387+0000","last_active":"2024-04-25T01:08:29.827333+0000","last_peered":"2024-04-25T01:08:29.827333+0000","last_clean":"2024-04-25T01:08:29.827333+0000","last_became_active":"2024-04-25T01:08:24.524324+0000","last_became_peered":"2024-04-25T01:08:24.524324+0000","last_unstale":"2024-04-25T01:08:29.827333+0000","last_undegraded":"2024-04-25T01:08:29.827333+0000","last_fullsized":"2024-04-25T01:08:29.827333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:49.276483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":102463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":102553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":693469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:08:42.211 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:08:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.381+0000 7f1939d1a640 1 Processor -- start 2024-04-25T01:08:42.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.381+0000 7f1939d1a640 1 -- start start 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1939d1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f193414bb50 0x7f19340cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1939d1a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f193406dab0 0x7f193406df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1939d1a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193414bf30 con 0x7f1934070d20 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1939d1a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1934072d40 con 0x7f193406dab0 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1939d1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f193406e450 con 0x7f193414bb50 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1932ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f193406dab0 0x7f193406df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1933fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1934070d20 0x7f19340b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33848/0 (socket says 172.21.15.33:33848) 2024-04-25T01:08:42.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1933fff640 1 -- 172.21.15.33:0/1399012577 learned_addr learned my addr 172.21.15.33:0/1399012577 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f19337fe640 1 --2- 172.21.15.33:0/1399012577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f193414bb50 0x7f19340cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1932ffd640 1 -- 172.21.15.33:0/1399012577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f193414bb50 msgr2=0x7f19340cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1932ffd640 1 --2- 172.21.15.33:0/1399012577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f193414bb50 0x7f19340cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f19327fc640 1 -- 172.21.15.33:0/1399012577 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3800247241 0 0) 0x7f193414bf30 con 0x7f1934070d20 2024-04-25T01:08:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1932ffd640 1 -- 172.21.15.33:0/1399012577 >> v1:172.21.15.33:6789/0 conn(0x7f1934070d20 legacy=0x7f19340b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1932ffd640 1 -- 172.21.15.33:0/1399012577 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1934071900 con 0x7f193406dab0 2024-04-25T01:08:42.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.385+0000 7f1932ffd640 1 --2- 172.21.15.33:0/1399012577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f193406df10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f191c0097f0 tx=0x7f191c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e374ee2495ecd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:42.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19327fc640 1 -- 172.21.15.33:0/1399012577 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c0b0070 con 0x7f193406dab0 2024-04-25T01:08:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19327fc640 1 -- 172.21.15.33:0/1399012577 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f191c0ab600 con 0x7f193406dab0 2024-04-25T01:08:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19327fc640 1 -- 172.21.15.33:0/1399012577 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191c0ab9d0 con 0x7f193406dab0 2024-04-25T01:08:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- 172.21.15.33:0/1399012577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 msgr2=0x7f193406df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 --2- 172.21.15.33:0/1399012577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f193406df10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f191c0097f0 tx=0x7f191c01c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- 172.21.15.33:0/1399012577 shutdown_connections 2024-04-25T01:08:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 --2- 172.21.15.33:0/1399012577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f193406df10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 --2- 172.21.15.33:0/1399012577 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f193414bb50 0x7f19340cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- 172.21.15.33:0/1399012577 >> 172.21.15.33:0/1399012577 conn(0x7f19340b1810 msgr2=0x7f19340b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- 172.21.15.33:0/1399012577 shutdown_connections 2024-04-25T01:08:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- 172.21.15.33:0/1399012577 wait complete. 2024-04-25T01:08:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 Processor -- start 2024-04-25T01:08:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- start start 2024-04-25T01:08:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f19340a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1934070d20 0x7f1934141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193414bb50 0x7f1934161290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f19340b4080 con 0x7f193406dab0 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1934071580 con 0x7f193414bb50 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f19340b4200 con 0x7f1934070d20 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f19340a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f19340a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49866/0 (socket says 172.21.15.33:49866) 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 -- 172.21.15.33:0/3146829770 learned_addr learned my addr 172.21.15.33:0/3146829770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1933fff640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193414bb50 0x7f1934161290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1932ffd640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1934070d20 0x7f1934141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 -- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1934070d20 msgr2=0x7f1934141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1934070d20 0x7f1934141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 -- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193414bb50 msgr2=0x7f1934161290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193414bb50 0x7f1934161290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 -- 172.21.15.33:0/3146829770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19340a9970 con 0x7f193406dab0 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f19337fe640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f19340a9430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f192c007ef0 tx=0x7f192c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f192bfff640 1 -- 172.21.15.33:0/3146829770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c014620 con 0x7f193406dab0 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f192bfff640 1 -- 172.21.15.33:0/3146829770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c019070 con 0x7f193406dab0 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19341423b0 con 0x7f193406dab0 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f192bfff640 1 -- 172.21.15.33:0/3146829770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c014a00 con 0x7f193406dab0 2024-04-25T01:08:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.389+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341476e0 con 0x7f193406dab0 2024-04-25T01:08:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.393+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1934069040 con 0x7f193406dab0 2024-04-25T01:08:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.393+0000 7f192bfff640 1 -- 172.21.15.33:0/3146829770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f192c014ba0 con 0x7f193406dab0 2024-04-25T01:08:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.393+0000 7f192bfff640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19140678c0 0x7f1914069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.393+0000 7f192bfff640 1 -- 172.21.15.33:0/3146829770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f192c086880 con 0x7f193406dab0 2024-04-25T01:08:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.393+0000 7f1932ffd640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19140678c0 0x7f1914069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.393+0000 7f1932ffd640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19140678c0 0x7f1914069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f191c0ad000 tx=0x7f191c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.397+0000 7f192bfff640 1 -- 172.21.15.33:0/3146829770 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f192c057110 con 0x7f193406dab0 2024-04-25T01:08:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.549+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f193406edd0 con 0x7f19140678c0 2024-04-25T01:08:42.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.573+0000 7f192bfff640 1 -- 172.21.15.33:0/3146829770 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332570 (secure 0 0 0) 0x7f193406edd0 con 0x7f19140678c0 2024-04-25T01:08:42.568 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:42.570 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19140678c0 msgr2=0x7f1914069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19140678c0 0x7f1914069d80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f191c0ad000 tx=0x7f191c0a6000 comp rx=0 tx=0).stop 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 msgr2=0x7f19340a9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f19340a9430 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f192c007ef0 tx=0x7f192c00e4b0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 shutdown_connections 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f19140678c0 0x7f1914069d80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f193414bb50 0x7f1934161290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1934070d20 0x7f1934141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 --2- 172.21.15.33:0/3146829770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f193406dab0 0x7f19340a9430 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 >> 172.21.15.33:0/3146829770 conn(0x7f19340b1810 msgr2=0x7f1934149980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 shutdown_connections 2024-04-25T01:08:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.577+0000 7f1939d1a640 1 -- 172.21.15.33:0/3146829770 wait complete. 2024-04-25T01:08:42.586 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-04-25T01:08:41.091571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223588,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751132,"statfs":{"total":773094113280,"available":772865159168,"internally_reserved":0,"allocated":3317760,"data_stored":2009438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.585087"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830172+0000","last_change":"2024-04-25T01:08:23.464284+0000","last_active":"2024-04-25T01:08:29.830172+0000","last_peered":"2024-04-25T01:08:29.830172+0000","last_clean":"2024-04-25T01:08:29.830172+0000","last_became_active":"2024-04-25T01:08:23.464018+0000","last_became_peered":"2024-04-25T01:08:23.464018+0000","last_unstale":"2024-04-25T01:08:29.830172+0000","last_undegraded":"2024-04-25T01:08:29.830172+0000","last_fullsized":"2024-04-25T01:08:29.830172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:27.195443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509770+0000","last_change":"2024-04-25T01:08:24.479420+0000","last_active":"2024-04-25T01:08:29.509770+0000","last_peered":"2024-04-25T01:08:29.509770+0000","last_clean":"2024-04-25T01:08:29.509770+0000","last_became_active":"2024-04-25T01:08:24.479003+0000","last_became_peered":"2024-04-25T01:08:24.479003+0000","last_unstale":"2024-04-25T01:08:29.509770+0000","last_undegraded":"2024-04-25T01:08:29.509770+0000","last_fullsized":"2024-04-25T01:08:29.509770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:10.309810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507734+0000","last_change":"2024-04-25T01:08:28.865743+0000","last_active":"2024-04-25T01:08:29.507734+0000","last_peered":"2024-04-25T01:08:29.507734+0000","last_clean":"2024-04-25T01:08:29.507734+0000","last_became_active":"2024-04-25T01:08:28.864434+0000","last_became_peered":"2024-04-25T01:08:28.864434+0000","last_unstale":"2024-04-25T01:08:29.507734+0000","last_undegraded":"2024-04-25T01:08:29.507734+0000","last_fullsized":"2024-04-25T01:08:29.507734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.379931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510114+0000","last_change":"2024-04-25T01:08:23.447876+0000","last_active":"2024-04-25T01:08:29.510114+0000","last_peered":"2024-04-25T01:08:29.510114+0000","last_clean":"2024-04-25T01:08:29.510114+0000","last_became_active":"2024-04-25T01:08:23.447588+0000","last_became_peered":"2024-04-25T01:08:23.447588+0000","last_unstale":"2024-04-25T01:08:29.510114+0000","last_undegraded":"2024-04-25T01:08:29.510114+0000","last_fullsized":"2024-04-25T01:08:29.510114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830021+0000","last_change":"2024-04-25T01:08:24.494422+0000","last_active":"2024-04-25T01:08:29.830021+0000","last_peered":"2024-04-25T01:08:29.830021+0000","last_clean":"2024-04-25T01:08:29.830021+0000","last_became_active":"2024-04-25T01:08:24.494035+0000","last_became_peered":"2024-04-25T01:08:24.494035+0000","last_unstale":"2024-04-25T01:08:29.830021+0000","last_undegraded":"2024-04-25T01:08:29.830021+0000","last_fullsized":"2024-04-25T01:08:29.830021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510126+0000","last_change":"2024-04-25T01:08:28.548777+0000","last_active":"2024-04-25T01:08:29.510126+0000","last_peered":"2024-04-25T01:08:29.510126+0000","last_clean":"2024-04-25T01:08:29.510126+0000","last_became_active":"2024-04-25T01:08:28.545213+0000","last_became_peered":"2024-04-25T01:08:28.545213+0000","last_unstale":"2024-04-25T01:08:29.510126+0000","last_undegraded":"2024-04-25T01:08:29.510126+0000","last_fullsized":"2024-04-25T01:08:29.510126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:14.372448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826296+0000","last_change":"2024-04-25T01:08:23.483744+0000","last_active":"2024-04-25T01:08:29.826296+0000","last_peered":"2024-04-25T01:08:29.826296+0000","last_clean":"2024-04-25T01:08:29.826296+0000","last_became_active":"2024-04-25T01:08:23.483245+0000","last_became_peered":"2024-04-25T01:08:23.483245+0000","last_unstale":"2024-04-25T01:08:29.826296+0000","last_undegraded":"2024-04-25T01:08:29.826296+0000","last_fullsized":"2024-04-25T01:08:29.826296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:38.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826404+0000","last_change":"2024-04-25T01:08:24.480206+0000","last_active":"2024-04-25T01:08:29.826404+0000","last_peered":"2024-04-25T01:08:29.826404+0000","last_clean":"2024-04-25T01:08:29.826404+0000","last_became_active":"2024-04-25T01:08:24.479327+0000","last_became_peered":"2024-04-25T01:08:24.479327+0000","last_unstale":"2024-04-25T01:08:29.826404+0000","last_undegraded":"2024-04-25T01:08:29.826404+0000","last_fullsized":"2024-04-25T01:08:29.826404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.885275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510631+0000","last_change":"2024-04-25T01:08:28.545701+0000","last_active":"2024-04-25T01:08:29.510631+0000","last_peered":"2024-04-25T01:08:29.510631+0000","last_clean":"2024-04-25T01:08:29.510631+0000","last_became_active":"2024-04-25T01:08:28.544731+0000","last_became_peered":"2024-04-25T01:08:28.544731+0000","last_unstale":"2024-04-25T01:08:29.510631+0000","last_undegraded":"2024-04-25T01:08:29.510631+0000","last_fullsized":"2024-04-25T01:08:29.510631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.483124+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.482403+0000","last_became_peered":"2024-04-25T01:08:23.482403+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:22.095395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827570+0000","last_change":"2024-04-25T01:08:24.488697+0000","last_active":"2024-04-25T01:08:29.827570+0000","last_peered":"2024-04-25T01:08:29.827570+0000","last_clean":"2024-04-25T01:08:29.827570+0000","last_became_active":"2024-04-25T01:08:24.487850+0000","last_became_peered":"2024-04-25T01:08:24.487850+0000","last_unstale":"2024-04-25T01:08:29.827570+0000","last_undegraded":"2024-04-25T01:08:29.827570+0000","last_fullsized":"2024-04-25T01:08:29.827570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:57.185281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510492+0000","last_change":"2024-04-25T01:08:28.545916+0000","last_active":"2024-04-25T01:08:29.510492+0000","last_peered":"2024-04-25T01:08:29.510492+0000","last_clean":"2024-04-25T01:08:29.510492+0000","last_became_active":"2024-04-25T01:08:28.544666+0000","last_became_peered":"2024-04-25T01:08:28.544666+0000","last_unstale":"2024-04-25T01:08:29.510492+0000","last_undegraded":"2024-04-25T01:08:29.510492+0000","last_fullsized":"2024-04-25T01:08:29.510492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:03.657733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827605+0000","last_change":"2024-04-25T01:08:24.498408+0000","last_active":"2024-04-25T01:08:29.827605+0000","last_peered":"2024-04-25T01:08:29.827605+0000","last_clean":"2024-04-25T01:08:29.827605+0000","last_became_active":"2024-04-25T01:08:24.497888+0000","last_became_peered":"2024-04-25T01:08:24.497888+0000","last_unstale":"2024-04-25T01:08:29.827605+0000","last_undegraded":"2024-04-25T01:08:29.827605+0000","last_fullsized":"2024-04-25T01:08:29.827605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828026+0000","last_change":"2024-04-25T01:08:23.482207+0000","last_active":"2024-04-25T01:08:29.828026+0000","last_peered":"2024-04-25T01:08:29.828026+0000","last_clean":"2024-04-25T01:08:29.828026+0000","last_became_active":"2024-04-25T01:08:23.475806+0000","last_became_peered":"2024-04-25T01:08:23.475806+0000","last_unstale":"2024-04-25T01:08:29.828026+0000","last_undegraded":"2024-04-25T01:08:29.828026+0000","last_fullsized":"2024-04-25T01:08:29.828026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:56.775800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512324+0000","last_change":"2024-04-25T01:08:28.537642+0000","last_active":"2024-04-25T01:08:29.512324+0000","last_peered":"2024-04-25T01:08:29.512324+0000","last_clean":"2024-04-25T01:08:29.512324+0000","last_became_active":"2024-04-25T01:08:28.536434+0000","last_became_peered":"2024-04-25T01:08:28.536434+0000","last_unstale":"2024-04-25T01:08:29.512324+0000","last_undegraded":"2024-04-25T01:08:29.512324+0000","last_fullsized":"2024-04-25T01:08:29.512324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508548+0000","last_change":"2024-04-25T01:08:24.497990+0000","last_active":"2024-04-25T01:08:29.508548+0000","last_peered":"2024-04-25T01:08:29.508548+0000","last_clean":"2024-04-25T01:08:29.508548+0000","last_became_active":"2024-04-25T01:08:24.497701+0000","last_became_peered":"2024-04-25T01:08:24.497701+0000","last_unstale":"2024-04-25T01:08:29.508548+0000","last_undegraded":"2024-04-25T01:08:29.508548+0000","last_fullsized":"2024-04-25T01:08:29.508548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:25.853831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.448779+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.448310+0000","last_became_peered":"2024-04-25T01:08:23.448310+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510707+0000","last_change":"2024-04-25T01:08:28.546163+0000","last_active":"2024-04-25T01:08:29.510707+0000","last_peered":"2024-04-25T01:08:29.510707+0000","last_clean":"2024-04-25T01:08:29.510707+0000","last_became_active":"2024-04-25T01:08:28.545022+0000","last_became_peered":"2024-04-25T01:08:28.545022+0000","last_unstale":"2024-04-25T01:08:29.510707+0000","last_undegraded":"2024-04-25T01:08:29.510707+0000","last_fullsized":"2024-04-25T01:08:29.510707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508535+0000","last_change":"2024-04-25T01:08:24.472427+0000","last_active":"2024-04-25T01:08:29.508535+0000","last_peered":"2024-04-25T01:08:29.508535+0000","last_clean":"2024-04-25T01:08:29.508535+0000","last_became_active":"2024-04-25T01:08:24.471579+0000","last_became_peered":"2024-04-25T01:08:24.471579+0000","last_unstale":"2024-04-25T01:08:29.508535+0000","last_undegraded":"2024-04-25T01:08:29.508535+0000","last_fullsized":"2024-04-25T01:08:29.508535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828376+0000","last_change":"2024-04-25T01:08:23.475602+0000","last_active":"2024-04-25T01:08:29.828376+0000","last_peered":"2024-04-25T01:08:29.828376+0000","last_clean":"2024-04-25T01:08:29.828376+0000","last_became_active":"2024-04-25T01:08:23.475273+0000","last_became_peered":"2024-04-25T01:08:23.475273+0000","last_unstale":"2024-04-25T01:08:29.828376+0000","last_undegraded":"2024-04-25T01:08:29.828376+0000","last_fullsized":"2024-04-25T01:08:29.828376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830145+0000","last_change":"2024-04-25T01:08:28.858908+0000","last_active":"2024-04-25T01:08:29.830145+0000","last_peered":"2024-04-25T01:08:29.830145+0000","last_clean":"2024-04-25T01:08:29.830145+0000","last_became_active":"2024-04-25T01:08:28.858052+0000","last_became_peered":"2024-04-25T01:08:28.858052+0000","last_unstale":"2024-04-25T01:08:29.830145+0000","last_undegraded":"2024-04-25T01:08:29.830145+0000","last_fullsized":"2024-04-25T01:08:29.830145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:53.339121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511155+0000","last_change":"2024-04-25T01:08:24.473202+0000","last_active":"2024-04-25T01:08:29.511155+0000","last_peered":"2024-04-25T01:08:29.511155+0000","last_clean":"2024-04-25T01:08:29.511155+0000","last_became_active":"2024-04-25T01:08:24.470580+0000","last_became_peered":"2024-04-25T01:08:24.470580+0000","last_unstale":"2024-04-25T01:08:29.511155+0000","last_undegraded":"2024-04-25T01:08:29.511155+0000","last_fullsized":"2024-04-25T01:08:29.511155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:27.238691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828899+0000","last_change":"2024-04-25T01:08:23.482632+0000","last_active":"2024-04-25T01:08:29.828899+0000","last_peered":"2024-04-25T01:08:29.828899+0000","last_clean":"2024-04-25T01:08:29.828899+0000","last_became_active":"2024-04-25T01:08:23.482050+0000","last_became_peered":"2024-04-25T01:08:23.482050+0000","last_unstale":"2024-04-25T01:08:29.828899+0000","last_undegraded":"2024-04-25T01:08:29.828899+0000","last_fullsized":"2024-04-25T01:08:29.828899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512530+0000","last_change":"2024-04-25T01:08:28.538805+0000","last_active":"2024-04-25T01:08:29.512530+0000","last_peered":"2024-04-25T01:08:29.512530+0000","last_clean":"2024-04-25T01:08:29.512530+0000","last_became_active":"2024-04-25T01:08:28.537512+0000","last_became_peered":"2024-04-25T01:08:28.537512+0000","last_unstale":"2024-04-25T01:08:29.512530+0000","last_undegraded":"2024-04-25T01:08:29.512530+0000","last_fullsized":"2024-04-25T01:08:29.512530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829865+0000","last_change":"2024-04-25T01:08:24.488971+0000","last_active":"2024-04-25T01:08:29.829865+0000","last_peered":"2024-04-25T01:08:29.829865+0000","last_clean":"2024-04-25T01:08:29.829865+0000","last_became_active":"2024-04-25T01:08:24.488318+0000","last_became_peered":"2024-04-25T01:08:24.488318+0000","last_unstale":"2024-04-25T01:08:29.829865+0000","last_undegraded":"2024-04-25T01:08:29.829865+0000","last_fullsized":"2024-04-25T01:08:29.829865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:48.906624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829517+0000","last_change":"2024-04-25T01:08:23.484304+0000","last_active":"2024-04-25T01:08:29.829517+0000","last_peered":"2024-04-25T01:08:29.829517+0000","last_clean":"2024-04-25T01:08:29.829517+0000","last_became_active":"2024-04-25T01:08:23.483396+0000","last_became_peered":"2024-04-25T01:08:23.483396+0000","last_unstale":"2024-04-25T01:08:29.829517+0000","last_undegraded":"2024-04-25T01:08:29.829517+0000","last_fullsized":"2024-04-25T01:08:29.829517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510293+0000","last_change":"2024-04-25T01:08:28.861041+0000","last_active":"2024-04-25T01:08:29.510293+0000","last_peered":"2024-04-25T01:08:29.510293+0000","last_clean":"2024-04-25T01:08:29.510293+0000","last_became_active":"2024-04-25T01:08:28.859853+0000","last_became_peered":"2024-04-25T01:08:28.859853+0000","last_unstale":"2024-04-25T01:08:29.510293+0000","last_undegraded":"2024-04-25T01:08:29.510293+0000","last_fullsized":"2024-04-25T01:08:29.510293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:58.665747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510472+0000","last_change":"2024-04-25T01:08:24.513917+0000","last_active":"2024-04-25T01:08:29.510472+0000","last_peered":"2024-04-25T01:08:29.510472+0000","last_clean":"2024-04-25T01:08:29.510472+0000","last_became_active":"2024-04-25T01:08:24.513551+0000","last_became_peered":"2024-04-25T01:08:24.513551+0000","last_unstale":"2024-04-25T01:08:29.510472+0000","last_undegraded":"2024-04-25T01:08:29.510472+0000","last_fullsized":"2024-04-25T01:08:29.510472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829765+0000","last_change":"2024-04-25T01:08:23.463105+0000","last_active":"2024-04-25T01:08:29.829765+0000","last_peered":"2024-04-25T01:08:29.829765+0000","last_clean":"2024-04-25T01:08:29.829765+0000","last_became_active":"2024-04-25T01:08:23.462766+0000","last_became_peered":"2024-04-25T01:08:23.462766+0000","last_unstale":"2024-04-25T01:08:29.829765+0000","last_undegraded":"2024-04-25T01:08:29.829765+0000","last_fullsized":"2024-04-25T01:08:29.829765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507606+0000","last_change":"2024-04-25T01:08:28.859248+0000","last_active":"2024-04-25T01:08:29.507606+0000","last_peered":"2024-04-25T01:08:29.507606+0000","last_clean":"2024-04-25T01:08:29.507606+0000","last_became_active":"2024-04-25T01:08:28.858134+0000","last_became_peered":"2024-04-25T01:08:28.858134+0000","last_unstale":"2024-04-25T01:08:29.507606+0000","last_undegraded":"2024-04-25T01:08:29.507606+0000","last_fullsized":"2024-04-25T01:08:29.507606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:54.239410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827142+0000","last_change":"2024-04-25T01:08:24.463169+0000","last_active":"2024-04-25T01:08:29.827142+0000","last_peered":"2024-04-25T01:08:29.827142+0000","last_clean":"2024-04-25T01:08:29.827142+0000","last_became_active":"2024-04-25T01:08:24.462836+0000","last_became_peered":"2024-04-25T01:08:24.462836+0000","last_unstale":"2024-04-25T01:08:29.827142+0000","last_undegraded":"2024-04-25T01:08:29.827142+0000","last_fullsized":"2024-04-25T01:08:29.827142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826900+0000","last_change":"2024-04-25T01:08:23.482862+0000","last_active":"2024-04-25T01:08:29.826900+0000","last_peered":"2024-04-25T01:08:29.826900+0000","last_clean":"2024-04-25T01:08:29.826900+0000","last_became_active":"2024-04-25T01:08:23.482148+0000","last_became_peered":"2024-04-25T01:08:23.482148+0000","last_unstale":"2024-04-25T01:08:29.826900+0000","last_undegraded":"2024-04-25T01:08:29.826900+0000","last_fullsized":"2024-04-25T01:08:29.826900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:38.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826923+0000","last_change":"2024-04-25T01:08:28.853689+0000","last_active":"2024-04-25T01:08:29.826923+0000","last_peered":"2024-04-25T01:08:29.826923+0000","last_clean":"2024-04-25T01:08:29.826923+0000","last_became_active":"2024-04-25T01:08:28.852691+0000","last_became_peered":"2024-04-25T01:08:28.852691+0000","last_unstale":"2024-04-25T01:08:29.826923+0000","last_undegraded":"2024-04-25T01:08:29.826923+0000","last_fullsized":"2024-04-25T01:08:29.826923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:01.247919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828543+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.828543+0000","last_peered":"2024-04-25T01:08:29.828543+0000","last_clean":"2024-04-25T01:08:29.828543+0000","last_became_active":"2024-04-25T01:08:24.489833+0000","last_became_peered":"2024-04-25T01:08:24.489833+0000","last_unstale":"2024-04-25T01:08:29.828543+0000","last_undegraded":"2024-04-25T01:08:29.828543+0000","last_fullsized":"2024-04-25T01:08:29.828543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:43.847545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:23.451486+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:23.451147+0000","last_became_peered":"2024-04-25T01:08:23.451147+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824912+0000","last_change":"2024-04-25T01:08:28.546785+0000","last_active":"2024-04-25T01:08:29.824912+0000","last_peered":"2024-04-25T01:08:29.824912+0000","last_clean":"2024-04-25T01:08:29.824912+0000","last_became_active":"2024-04-25T01:08:28.546088+0000","last_became_peered":"2024-04-25T01:08:28.546088+0000","last_unstale":"2024-04-25T01:08:29.824912+0000","last_undegraded":"2024-04-25T01:08:29.824912+0000","last_fullsized":"2024-04-25T01:08:29.824912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511711+0000","last_change":"2024-04-25T01:08:24.469958+0000","last_active":"2024-04-25T01:08:29.511711+0000","last_peered":"2024-04-25T01:08:29.511711+0000","last_clean":"2024-04-25T01:08:29.511711+0000","last_became_active":"2024-04-25T01:08:24.468421+0000","last_became_peered":"2024-04-25T01:08:24.468421+0000","last_unstale":"2024-04-25T01:08:29.511711+0000","last_undegraded":"2024-04-25T01:08:29.511711+0000","last_fullsized":"2024-04-25T01:08:29.511711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:37.314427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826336+0000","last_change":"2024-04-25T01:08:23.485610+0000","last_active":"2024-04-25T01:08:29.826336+0000","last_peered":"2024-04-25T01:08:29.826336+0000","last_clean":"2024-04-25T01:08:29.826336+0000","last_became_active":"2024-04-25T01:08:23.484885+0000","last_became_peered":"2024-04-25T01:08:23.484885+0000","last_unstale":"2024-04-25T01:08:29.826336+0000","last_undegraded":"2024-04-25T01:08:29.826336+0000","last_fullsized":"2024-04-25T01:08:29.826336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:41.214012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826899+0000","last_change":"2024-04-25T01:08:28.541717+0000","last_active":"2024-04-25T01:08:29.826899+0000","last_peered":"2024-04-25T01:08:29.826899+0000","last_clean":"2024-04-25T01:08:29.826899+0000","last_became_active":"2024-04-25T01:08:28.541136+0000","last_became_peered":"2024-04-25T01:08:28.541136+0000","last_unstale":"2024-04-25T01:08:29.826899+0000","last_undegraded":"2024-04-25T01:08:29.826899+0000","last_fullsized":"2024-04-25T01:08:29.826899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824739+0000","last_change":"2024-04-25T01:08:24.468504+0000","last_active":"2024-04-25T01:08:29.824739+0000","last_peered":"2024-04-25T01:08:29.824739+0000","last_clean":"2024-04-25T01:08:29.824739+0000","last_became_active":"2024-04-25T01:08:24.468220+0000","last_became_peered":"2024-04-25T01:08:24.468220+0000","last_unstale":"2024-04-25T01:08:29.824739+0000","last_undegraded":"2024-04-25T01:08:29.824739+0000","last_fullsized":"2024-04-25T01:08:29.824739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825079+0000","last_change":"2024-04-25T01:08:23.448344+0000","last_active":"2024-04-25T01:08:29.825079+0000","last_peered":"2024-04-25T01:08:29.825079+0000","last_clean":"2024-04-25T01:08:29.825079+0000","last_became_active":"2024-04-25T01:08:23.447926+0000","last_became_peered":"2024-04-25T01:08:23.447926+0000","last_unstale":"2024-04-25T01:08:29.825079+0000","last_undegraded":"2024-04-25T01:08:29.825079+0000","last_fullsized":"2024-04-25T01:08:29.825079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830344+0000","last_change":"2024-04-25T01:08:28.547149+0000","last_active":"2024-04-25T01:08:29.830344+0000","last_peered":"2024-04-25T01:08:29.830344+0000","last_clean":"2024-04-25T01:08:29.830344+0000","last_became_active":"2024-04-25T01:08:28.546228+0000","last_became_peered":"2024-04-25T01:08:28.546228+0000","last_unstale":"2024-04-25T01:08:29.830344+0000","last_undegraded":"2024-04-25T01:08:29.830344+0000","last_fullsized":"2024-04-25T01:08:29.830344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.235046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825749+0000","last_change":"2024-04-25T01:08:24.458889+0000","last_active":"2024-04-25T01:08:29.825749+0000","last_peered":"2024-04-25T01:08:29.825749+0000","last_clean":"2024-04-25T01:08:29.825749+0000","last_became_active":"2024-04-25T01:08:24.458591+0000","last_became_peered":"2024-04-25T01:08:24.458591+0000","last_unstale":"2024-04-25T01:08:29.825749+0000","last_undegraded":"2024-04-25T01:08:29.825749+0000","last_fullsized":"2024-04-25T01:08:29.825749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:40.997594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829116+0000","last_change":"2024-04-25T01:08:23.481327+0000","last_active":"2024-04-25T01:08:29.829116+0000","last_peered":"2024-04-25T01:08:29.829116+0000","last_clean":"2024-04-25T01:08:29.829116+0000","last_became_active":"2024-04-25T01:08:23.475537+0000","last_became_peered":"2024-04-25T01:08:23.475537+0000","last_unstale":"2024-04-25T01:08:29.829116+0000","last_undegraded":"2024-04-25T01:08:29.829116+0000","last_fullsized":"2024-04-25T01:08:29.829116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829514+0000","last_change":"2024-04-25T01:08:28.859351+0000","last_active":"2024-04-25T01:08:29.829514+0000","last_peered":"2024-04-25T01:08:29.829514+0000","last_clean":"2024-04-25T01:08:29.829514+0000","last_became_active":"2024-04-25T01:08:28.858099+0000","last_became_peered":"2024-04-25T01:08:28.858099+0000","last_unstale":"2024-04-25T01:08:29.829514+0000","last_undegraded":"2024-04-25T01:08:29.829514+0000","last_fullsized":"2024-04-25T01:08:29.829514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:46.276943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508474+0000","last_change":"2024-04-25T01:08:24.495211+0000","last_active":"2024-04-25T01:08:29.508474+0000","last_peered":"2024-04-25T01:08:29.508474+0000","last_clean":"2024-04-25T01:08:29.508474+0000","last_became_active":"2024-04-25T01:08:24.494930+0000","last_became_peered":"2024-04-25T01:08:24.494930+0000","last_unstale":"2024-04-25T01:08:29.508474+0000","last_undegraded":"2024-04-25T01:08:29.508474+0000","last_fullsized":"2024-04-25T01:08:29.508474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825747+0000","last_change":"2024-04-25T01:08:23.461624+0000","last_active":"2024-04-25T01:08:29.825747+0000","last_peered":"2024-04-25T01:08:29.825747+0000","last_clean":"2024-04-25T01:08:29.825747+0000","last_became_active":"2024-04-25T01:08:23.457067+0000","last_became_peered":"2024-04-25T01:08:23.457067+0000","last_unstale":"2024-04-25T01:08:29.825747+0000","last_undegraded":"2024-04-25T01:08:29.825747+0000","last_fullsized":"2024-04-25T01:08:29.825747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829168+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:08:29.829168+0000","last_peered":"2024-04-25T01:08:29.829168+0000","last_clean":"2024-04-25T01:08:29.829168+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:08:29.829168+0000","last_undegraded":"2024-04-25T01:08:29.829168+0000","last_fullsized":"2024-04-25T01:08:29.829168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826455+0000","last_change":"2024-04-25T01:08:28.861776+0000","last_active":"2024-04-25T01:08:29.826455+0000","last_peered":"2024-04-25T01:08:29.826455+0000","last_clean":"2024-04-25T01:08:29.826455+0000","last_became_active":"2024-04-25T01:08:28.860715+0000","last_became_peered":"2024-04-25T01:08:28.860715+0000","last_unstale":"2024-04-25T01:08:29.826455+0000","last_undegraded":"2024-04-25T01:08:29.826455+0000","last_fullsized":"2024-04-25T01:08:29.826455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826011+0000","last_change":"2024-04-25T01:08:24.523631+0000","last_active":"2024-04-25T01:08:29.826011+0000","last_peered":"2024-04-25T01:08:29.826011+0000","last_clean":"2024-04-25T01:08:29.826011+0000","last_became_active":"2024-04-25T01:08:24.523130+0000","last_became_peered":"2024-04-25T01:08:24.523130+0000","last_unstale":"2024-04-25T01:08:29.826011+0000","last_undegraded":"2024-04-25T01:08:29.826011+0000","last_fullsized":"2024-04-25T01:08:29.826011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:19.861291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826698+0000","last_change":"2024-04-25T01:08:23.470284+0000","last_active":"2024-04-25T01:08:29.826698+0000","last_peered":"2024-04-25T01:08:29.826698+0000","last_clean":"2024-04-25T01:08:29.826698+0000","last_became_active":"2024-04-25T01:08:23.470043+0000","last_became_peered":"2024-04-25T01:08:23.470043+0000","last_unstale":"2024-04-25T01:08:29.826698+0000","last_undegraded":"2024-04-25T01:08:29.826698+0000","last_fullsized":"2024-04-25T01:08:29.826698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827347+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827347+0000","last_peered":"2024-04-25T01:08:29.827347+0000","last_clean":"2024-04-25T01:08:29.827347+0000","last_became_active":"2024-04-25T01:08:28.858545+0000","last_became_peered":"2024-04-25T01:08:28.858545+0000","last_unstale":"2024-04-25T01:08:29.827347+0000","last_undegraded":"2024-04-25T01:08:29.827347+0000","last_fullsized":"2024-04-25T01:08:29.827347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826582+0000","last_change":"2024-04-25T01:08:24.490983+0000","last_active":"2024-04-25T01:08:29.826582+0000","last_peered":"2024-04-25T01:08:29.826582+0000","last_clean":"2024-04-25T01:08:29.826582+0000","last_became_active":"2024-04-25T01:08:24.490276+0000","last_became_peered":"2024-04-25T01:08:24.490276+0000","last_unstale":"2024-04-25T01:08:29.826582+0000","last_undegraded":"2024-04-25T01:08:29.826582+0000","last_fullsized":"2024-04-25T01:08:29.826582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:17.470136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829394+0000","last_change":"2024-04-25T01:08:23.459773+0000","last_active":"2024-04-25T01:08:29.829394+0000","last_peered":"2024-04-25T01:08:29.829394+0000","last_clean":"2024-04-25T01:08:29.829394+0000","last_became_active":"2024-04-25T01:08:23.459374+0000","last_became_peered":"2024-04-25T01:08:23.459374+0000","last_unstale":"2024-04-25T01:08:29.829394+0000","last_undegraded":"2024-04-25T01:08:29.829394+0000","last_fullsized":"2024-04-25T01:08:29.829394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827646+0000","last_change":"2024-04-25T01:08:28.542952+0000","last_active":"2024-04-25T01:08:29.827646+0000","last_peered":"2024-04-25T01:08:29.827646+0000","last_clean":"2024-04-25T01:08:29.827646+0000","last_became_active":"2024-04-25T01:08:28.542388+0000","last_became_peered":"2024-04-25T01:08:28.542388+0000","last_unstale":"2024-04-25T01:08:29.827646+0000","last_undegraded":"2024-04-25T01:08:29.827646+0000","last_fullsized":"2024-04-25T01:08:29.827646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:47.910183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828070+0000","last_change":"2024-04-25T01:08:24.525832+0000","last_active":"2024-04-25T01:08:29.828070+0000","last_peered":"2024-04-25T01:08:29.828070+0000","last_clean":"2024-04-25T01:08:29.828070+0000","last_became_active":"2024-04-25T01:08:24.524963+0000","last_became_peered":"2024-04-25T01:08:24.524963+0000","last_unstale":"2024-04-25T01:08:29.828070+0000","last_undegraded":"2024-04-25T01:08:29.828070+0000","last_fullsized":"2024-04-25T01:08:29.828070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:54.172408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828701+0000","last_change":"2024-04-25T01:08:23.448056+0000","last_active":"2024-04-25T01:08:29.828701+0000","last_peered":"2024-04-25T01:08:29.828701+0000","last_clean":"2024-04-25T01:08:29.828701+0000","last_became_active":"2024-04-25T01:08:23.447748+0000","last_became_peered":"2024-04-25T01:08:23.447748+0000","last_unstale":"2024-04-25T01:08:29.828701+0000","last_undegraded":"2024-04-25T01:08:29.828701+0000","last_fullsized":"2024-04-25T01:08:29.828701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824750+0000","last_change":"2024-04-25T01:08:28.860380+0000","last_active":"2024-04-25T01:08:29.824750+0000","last_peered":"2024-04-25T01:08:29.824750+0000","last_clean":"2024-04-25T01:08:29.824750+0000","last_became_active":"2024-04-25T01:08:28.859396+0000","last_became_peered":"2024-04-25T01:08:28.859396+0000","last_unstale":"2024-04-25T01:08:29.824750+0000","last_undegraded":"2024-04-25T01:08:29.824750+0000","last_fullsized":"2024-04-25T01:08:29.824750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:28.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826570+0000","last_change":"2024-04-25T01:08:24.524589+0000","last_active":"2024-04-25T01:08:29.826570+0000","last_peered":"2024-04-25T01:08:29.826570+0000","last_clean":"2024-04-25T01:08:29.826570+0000","last_became_active":"2024-04-25T01:08:24.523861+0000","last_became_peered":"2024-04-25T01:08:24.523861+0000","last_unstale":"2024-04-25T01:08:29.826570+0000","last_undegraded":"2024-04-25T01:08:29.826570+0000","last_fullsized":"2024-04-25T01:08:29.826570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:53.746720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512670+0000","last_change":"2024-04-25T01:08:23.445365+0000","last_active":"2024-04-25T01:08:29.512670+0000","last_peered":"2024-04-25T01:08:29.512670+0000","last_clean":"2024-04-25T01:08:29.512670+0000","last_became_active":"2024-04-25T01:08:23.444505+0000","last_became_peered":"2024-04-25T01:08:23.444505+0000","last_unstale":"2024-04-25T01:08:29.512670+0000","last_undegraded":"2024-04-25T01:08:29.512670+0000","last_fullsized":"2024-04-25T01:08:29.512670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:51.459603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510009+0000","last_change":"2024-04-25T01:08:28.549416+0000","last_active":"2024-04-25T01:08:29.510009+0000","last_peered":"2024-04-25T01:08:29.510009+0000","last_clean":"2024-04-25T01:08:29.510009+0000","last_became_active":"2024-04-25T01:08:28.547126+0000","last_became_peered":"2024-04-25T01:08:28.547126+0000","last_unstale":"2024-04-25T01:08:29.510009+0000","last_undegraded":"2024-04-25T01:08:29.510009+0000","last_fullsized":"2024-04-25T01:08:29.510009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:29.916469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828474+0000","last_change":"2024-04-25T01:08:24.514329+0000","last_active":"2024-04-25T01:08:29.828474+0000","last_peered":"2024-04-25T01:08:29.828474+0000","last_clean":"2024-04-25T01:08:29.828474+0000","last_became_active":"2024-04-25T01:08:24.513670+0000","last_became_peered":"2024-04-25T01:08:24.513670+0000","last_unstale":"2024-04-25T01:08:29.828474+0000","last_undegraded":"2024-04-25T01:08:29.828474+0000","last_fullsized":"2024-04-25T01:08:29.828474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.823985+0000","last_change":"2024-04-25T01:08:23.449156+0000","last_active":"2024-04-25T01:08:29.823985+0000","last_peered":"2024-04-25T01:08:29.823985+0000","last_clean":"2024-04-25T01:08:29.823985+0000","last_became_active":"2024-04-25T01:08:23.448724+0000","last_became_peered":"2024-04-25T01:08:23.448724+0000","last_unstale":"2024-04-25T01:08:29.823985+0000","last_undegraded":"2024-04-25T01:08:29.823985+0000","last_fullsized":"2024-04-25T01:08:29.823985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829295+0000","last_change":"2024-04-25T01:08:28.538540+0000","last_active":"2024-04-25T01:08:29.829295+0000","last_peered":"2024-04-25T01:08:29.829295+0000","last_clean":"2024-04-25T01:08:29.829295+0000","last_became_active":"2024-04-25T01:08:28.537662+0000","last_became_peered":"2024-04-25T01:08:28.537662+0000","last_unstale":"2024-04-25T01:08:29.829295+0000","last_undegraded":"2024-04-25T01:08:29.829295+0000","last_fullsized":"2024-04-25T01:08:29.829295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:06.032667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824689+0000","last_change":"2024-04-25T01:08:24.499087+0000","last_active":"2024-04-25T01:08:29.824689+0000","last_peered":"2024-04-25T01:08:29.824689+0000","last_clean":"2024-04-25T01:08:29.824689+0000","last_became_active":"2024-04-25T01:08:24.498181+0000","last_became_peered":"2024-04-25T01:08:24.498181+0000","last_unstale":"2024-04-25T01:08:29.824689+0000","last_undegraded":"2024-04-25T01:08:29.824689+0000","last_fullsized":"2024-04-25T01:08:29.824689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825380+0000","last_change":"2024-04-25T01:08:23.466687+0000","last_active":"2024-04-25T01:08:29.825380+0000","last_peered":"2024-04-25T01:08:29.825380+0000","last_clean":"2024-04-25T01:08:29.825380+0000","last_became_active":"2024-04-25T01:08:23.465922+0000","last_became_peered":"2024-04-25T01:08:23.465922+0000","last_unstale":"2024-04-25T01:08:29.825380+0000","last_undegraded":"2024-04-25T01:08:29.825380+0000","last_fullsized":"2024-04-25T01:08:29.825380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510286+0000","last_change":"2024-04-25T01:08:28.861374+0000","last_active":"2024-04-25T01:08:29.510286+0000","last_peered":"2024-04-25T01:08:29.510286+0000","last_clean":"2024-04-25T01:08:29.510286+0000","last_became_active":"2024-04-25T01:08:28.857471+0000","last_became_peered":"2024-04-25T01:08:28.857471+0000","last_unstale":"2024-04-25T01:08:29.510286+0000","last_undegraded":"2024-04-25T01:08:29.510286+0000","last_fullsized":"2024-04-25T01:08:29.510286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:50.580633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827859+0000","last_change":"2024-04-25T01:08:24.524773+0000","last_active":"2024-04-25T01:08:29.827859+0000","last_peered":"2024-04-25T01:08:29.827859+0000","last_clean":"2024-04-25T01:08:29.827859+0000","last_became_active":"2024-04-25T01:08:24.523273+0000","last_became_peered":"2024-04-25T01:08:24.523273+0000","last_unstale":"2024-04-25T01:08:29.827859+0000","last_undegraded":"2024-04-25T01:08:29.827859+0000","last_fullsized":"2024-04-25T01:08:29.827859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:53.559257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826090+0000","last_change":"2024-04-25T01:08:23.478405+0000","last_active":"2024-04-25T01:08:29.826090+0000","last_peered":"2024-04-25T01:08:29.826090+0000","last_clean":"2024-04-25T01:08:29.826090+0000","last_became_active":"2024-04-25T01:08:23.477864+0000","last_became_peered":"2024-04-25T01:08:23.477864+0000","last_unstale":"2024-04-25T01:08:29.826090+0000","last_undegraded":"2024-04-25T01:08:29.826090+0000","last_fullsized":"2024-04-25T01:08:29.826090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:14.206151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509288+0000","last_change":"2024-04-25T01:08:28.859485+0000","last_active":"2024-04-25T01:08:29.509288+0000","last_peered":"2024-04-25T01:08:29.509288+0000","last_clean":"2024-04-25T01:08:29.509288+0000","last_became_active":"2024-04-25T01:08:28.858350+0000","last_became_peered":"2024-04-25T01:08:28.858350+0000","last_unstale":"2024-04-25T01:08:29.509288+0000","last_undegraded":"2024-04-25T01:08:29.509288+0000","last_fullsized":"2024-04-25T01:08:29.509288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:27.377063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:24.502358+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:24.502144+0000","last_became_peered":"2024-04-25T01:08:24.502144+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:17.614542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827872+0000","last_change":"2024-04-25T01:08:23.444558+0000","last_active":"2024-04-25T01:08:29.827872+0000","last_peered":"2024-04-25T01:08:29.827872+0000","last_clean":"2024-04-25T01:08:29.827872+0000","last_became_active":"2024-04-25T01:08:23.444246+0000","last_became_peered":"2024-04-25T01:08:23.444246+0000","last_unstale":"2024-04-25T01:08:29.827872+0000","last_undegraded":"2024-04-25T01:08:29.827872+0000","last_fullsized":"2024-04-25T01:08:29.827872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827499+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827499+0000","last_peered":"2024-04-25T01:08:29.827499+0000","last_clean":"2024-04-25T01:08:29.827499+0000","last_became_active":"2024-04-25T01:08:28.858324+0000","last_became_peered":"2024-04-25T01:08:28.858324+0000","last_unstale":"2024-04-25T01:08:29.827499+0000","last_undegraded":"2024-04-25T01:08:29.827499+0000","last_fullsized":"2024-04-25T01:08:29.827499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:49.074275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825751+0000","last_change":"2024-04-25T01:08:24.520516+0000","last_active":"2024-04-25T01:08:29.825751+0000","last_peered":"2024-04-25T01:08:29.825751+0000","last_clean":"2024-04-25T01:08:29.825751+0000","last_became_active":"2024-04-25T01:08:24.519967+0000","last_became_peered":"2024-04-25T01:08:24.519967+0000","last_unstale":"2024-04-25T01:08:29.825751+0000","last_undegraded":"2024-04-25T01:08:29.825751+0000","last_fullsized":"2024-04-25T01:08:29.825751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:50.176772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:33.334782+0000","last_change":"2024-04-25T01:08:23.460787+0000","last_active":"2024-04-25T01:08:33.334782+0000","last_peered":"2024-04-25T01:08:33.334782+0000","last_clean":"2024-04-25T01:08:33.334782+0000","last_became_active":"2024-04-25T01:08:23.455085+0000","last_became_peered":"2024-04-25T01:08:23.455085+0000","last_unstale":"2024-04-25T01:08:33.334782+0000","last_undegraded":"2024-04-25T01:08:33.334782+0000","last_fullsized":"2024-04-25T01:08:33.334782+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825396+0000","last_change":"2024-04-25T01:08:28.536251+0000","last_active":"2024-04-25T01:08:29.825396+0000","last_peered":"2024-04-25T01:08:29.825396+0000","last_clean":"2024-04-25T01:08:29.825396+0000","last_became_active":"2024-04-25T01:08:28.535359+0000","last_became_peered":"2024-04-25T01:08:28.535359+0000","last_unstale":"2024-04-25T01:08:29.825396+0000","last_undegraded":"2024-04-25T01:08:29.825396+0000","last_fullsized":"2024-04-25T01:08:29.825396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511615+0000","last_change":"2024-04-25T01:08:24.505163+0000","last_active":"2024-04-25T01:08:29.511615+0000","last_peered":"2024-04-25T01:08:29.511615+0000","last_clean":"2024-04-25T01:08:29.511615+0000","last_became_active":"2024-04-25T01:08:24.504390+0000","last_became_peered":"2024-04-25T01:08:24.504390+0000","last_unstale":"2024-04-25T01:08:29.511615+0000","last_undegraded":"2024-04-25T01:08:29.511615+0000","last_fullsized":"2024-04-25T01:08:29.511615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829299+0000","last_change":"2024-04-25T01:08:23.484553+0000","last_active":"2024-04-25T01:08:29.829299+0000","last_peered":"2024-04-25T01:08:29.829299+0000","last_clean":"2024-04-25T01:08:29.829299+0000","last_became_active":"2024-04-25T01:08:23.484289+0000","last_became_peered":"2024-04-25T01:08:23.484289+0000","last_unstale":"2024-04-25T01:08:29.829299+0000","last_undegraded":"2024-04-25T01:08:29.829299+0000","last_fullsized":"2024-04-25T01:08:29.829299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511206+0000","last_change":"2024-04-25T01:08:28.538547+0000","last_active":"2024-04-25T01:08:29.511206+0000","last_peered":"2024-04-25T01:08:29.511206+0000","last_clean":"2024-04-25T01:08:29.511206+0000","last_became_active":"2024-04-25T01:08:28.537497+0000","last_became_peered":"2024-04-25T01:08:28.537497+0000","last_unstale":"2024-04-25T01:08:29.511206+0000","last_undegraded":"2024-04-25T01:08:29.511206+0000","last_fullsized":"2024-04-25T01:08:29.511206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830135+0000","last_change":"2024-04-25T01:08:24.493527+0000","last_active":"2024-04-25T01:08:29.830135+0000","last_peered":"2024-04-25T01:08:29.830135+0000","last_clean":"2024-04-25T01:08:29.830135+0000","last_became_active":"2024-04-25T01:08:24.493206+0000","last_became_peered":"2024-04-25T01:08:24.493206+0000","last_unstale":"2024-04-25T01:08:29.830135+0000","last_undegraded":"2024-04-25T01:08:29.830135+0000","last_fullsized":"2024-04-25T01:08:29.830135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826688+0000","last_change":"2024-04-25T01:08:23.476900+0000","last_active":"2024-04-25T01:08:29.826688+0000","last_peered":"2024-04-25T01:08:29.826688+0000","last_clean":"2024-04-25T01:08:29.826688+0000","last_became_active":"2024-04-25T01:08:23.474893+0000","last_became_peered":"2024-04-25T01:08:23.474893+0000","last_unstale":"2024-04-25T01:08:29.826688+0000","last_undegraded":"2024-04-25T01:08:29.826688+0000","last_fullsized":"2024-04-25T01:08:29.826688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:25.208026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508202+0000","last_change":"2024-04-25T01:08:28.545577+0000","last_active":"2024-04-25T01:08:29.508202+0000","last_peered":"2024-04-25T01:08:29.508202+0000","last_clean":"2024-04-25T01:08:29.508202+0000","last_became_active":"2024-04-25T01:08:28.540902+0000","last_became_peered":"2024-04-25T01:08:28.540902+0000","last_unstale":"2024-04-25T01:08:29.508202+0000","last_undegraded":"2024-04-25T01:08:29.508202+0000","last_fullsized":"2024-04-25T01:08:29.508202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:14.257846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826495+0000","last_change":"2024-04-25T01:08:24.515829+0000","last_active":"2024-04-25T01:08:29.826495+0000","last_peered":"2024-04-25T01:08:29.826495+0000","last_clean":"2024-04-25T01:08:29.826495+0000","last_became_active":"2024-04-25T01:08:24.515267+0000","last_became_peered":"2024-04-25T01:08:24.515267+0000","last_unstale":"2024-04-25T01:08:29.826495+0000","last_undegraded":"2024-04-25T01:08:29.826495+0000","last_fullsized":"2024-04-25T01:08:29.826495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:51.870029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829322+0000","last_change":"2024-04-25T01:08:23.478857+0000","last_active":"2024-04-25T01:08:29.829322+0000","last_peered":"2024-04-25T01:08:29.829322+0000","last_clean":"2024-04-25T01:08:29.829322+0000","last_became_active":"2024-04-25T01:08:23.478215+0000","last_became_peered":"2024-04-25T01:08:23.478215+0000","last_unstale":"2024-04-25T01:08:29.829322+0000","last_undegraded":"2024-04-25T01:08:29.829322+0000","last_fullsized":"2024-04-25T01:08:29.829322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508255+0000","last_change":"2024-04-25T01:08:28.863219+0000","last_active":"2024-04-25T01:08:29.508255+0000","last_peered":"2024-04-25T01:08:29.508255+0000","last_clean":"2024-04-25T01:08:29.508255+0000","last_became_active":"2024-04-25T01:08:28.862202+0000","last_became_peered":"2024-04-25T01:08:28.862202+0000","last_unstale":"2024-04-25T01:08:29.508255+0000","last_undegraded":"2024-04-25T01:08:29.508255+0000","last_fullsized":"2024-04-25T01:08:29.508255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:58.057652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829042+0000","last_change":"2024-04-25T01:08:28.864029+0000","last_active":"2024-04-25T01:08:29.829042+0000","last_peered":"2024-04-25T01:08:29.829042+0000","last_clean":"2024-04-25T01:08:29.829042+0000","last_became_active":"2024-04-25T01:08:28.863040+0000","last_became_peered":"2024-04-25T01:08:28.863040+0000","last_unstale":"2024-04-25T01:08:29.829042+0000","last_undegraded":"2024-04-25T01:08:29.829042+0000","last_fullsized":"2024-04-25T01:08:29.829042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:55.932667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827904+0000","last_change":"2024-04-25T01:08:24.472094+0000","last_active":"2024-04-25T01:08:29.827904+0000","last_peered":"2024-04-25T01:08:29.827904+0000","last_clean":"2024-04-25T01:08:29.827904+0000","last_became_active":"2024-04-25T01:08:24.471381+0000","last_became_peered":"2024-04-25T01:08:24.471381+0000","last_unstale":"2024-04-25T01:08:29.827904+0000","last_undegraded":"2024-04-25T01:08:29.827904+0000","last_fullsized":"2024-04-25T01:08:29.827904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825540+0000","last_change":"2024-04-25T01:08:23.470504+0000","last_active":"2024-04-25T01:08:29.825540+0000","last_peered":"2024-04-25T01:08:29.825540+0000","last_clean":"2024-04-25T01:08:29.825540+0000","last_became_active":"2024-04-25T01:08:23.470060+0000","last_became_peered":"2024-04-25T01:08:23.470060+0000","last_unstale":"2024-04-25T01:08:29.825540+0000","last_undegraded":"2024-04-25T01:08:29.825540+0000","last_fullsized":"2024-04-25T01:08:29.825540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:52.050718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829192+0000","last_change":"2024-04-25T01:08:28.544590+0000","last_active":"2024-04-25T01:08:29.829192+0000","last_peered":"2024-04-25T01:08:29.829192+0000","last_clean":"2024-04-25T01:08:29.829192+0000","last_became_active":"2024-04-25T01:08:28.543195+0000","last_became_peered":"2024-04-25T01:08:28.543195+0000","last_unstale":"2024-04-25T01:08:29.829192+0000","last_undegraded":"2024-04-25T01:08:29.829192+0000","last_fullsized":"2024-04-25T01:08:29.829192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508183+0000","last_change":"2024-04-25T01:08:24.494258+0000","last_active":"2024-04-25T01:08:29.508183+0000","last_peered":"2024-04-25T01:08:29.508183+0000","last_clean":"2024-04-25T01:08:29.508183+0000","last_became_active":"2024-04-25T01:08:24.493935+0000","last_became_peered":"2024-04-25T01:08:24.493935+0000","last_unstale":"2024-04-25T01:08:29.508183+0000","last_undegraded":"2024-04-25T01:08:29.508183+0000","last_fullsized":"2024-04-25T01:08:29.508183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508536+0000","last_change":"2024-04-25T01:08:23.463618+0000","last_active":"2024-04-25T01:08:29.508536+0000","last_peered":"2024-04-25T01:08:29.508536+0000","last_clean":"2024-04-25T01:08:29.508536+0000","last_became_active":"2024-04-25T01:08:23.463303+0000","last_became_peered":"2024-04-25T01:08:23.463303+0000","last_unstale":"2024-04-25T01:08:29.508536+0000","last_undegraded":"2024-04-25T01:08:29.508536+0000","last_fullsized":"2024-04-25T01:08:29.508536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828071+0000","last_change":"2024-04-25T01:08:23.465959+0000","last_active":"2024-04-25T01:08:29.828071+0000","last_peered":"2024-04-25T01:08:29.828071+0000","last_clean":"2024-04-25T01:08:29.828071+0000","last_became_active":"2024-04-25T01:08:23.465518+0000","last_became_peered":"2024-04-25T01:08:23.465518+0000","last_unstale":"2024-04-25T01:08:29.828071+0000","last_undegraded":"2024-04-25T01:08:29.828071+0000","last_fullsized":"2024-04-25T01:08:29.828071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827782+0000","last_change":"2024-04-25T01:08:24.493189+0000","last_active":"2024-04-25T01:08:29.827782+0000","last_peered":"2024-04-25T01:08:29.827782+0000","last_clean":"2024-04-25T01:08:29.827782+0000","last_became_active":"2024-04-25T01:08:24.473740+0000","last_became_peered":"2024-04-25T01:08:24.473740+0000","last_unstale":"2024-04-25T01:08:29.827782+0000","last_undegraded":"2024-04-25T01:08:29.827782+0000","last_fullsized":"2024-04-25T01:08:29.827782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825925+0000","last_change":"2024-04-25T01:08:28.542530+0000","last_active":"2024-04-25T01:08:29.825925+0000","last_peered":"2024-04-25T01:08:29.825925+0000","last_clean":"2024-04-25T01:08:29.825925+0000","last_became_active":"2024-04-25T01:08:28.541445+0000","last_became_peered":"2024-04-25T01:08:28.541445+0000","last_unstale":"2024-04-25T01:08:29.825925+0000","last_undegraded":"2024-04-25T01:08:29.825925+0000","last_fullsized":"2024-04-25T01:08:29.825925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:21.963089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508697+0000","last_change":"2024-04-25T01:08:23.475915+0000","last_active":"2024-04-25T01:08:29.508697+0000","last_peered":"2024-04-25T01:08:29.508697+0000","last_clean":"2024-04-25T01:08:29.508697+0000","last_became_active":"2024-04-25T01:08:23.475628+0000","last_became_peered":"2024-04-25T01:08:23.475628+0000","last_unstale":"2024-04-25T01:08:29.508697+0000","last_undegraded":"2024-04-25T01:08:29.508697+0000","last_fullsized":"2024-04-25T01:08:29.508697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511101+0000","last_change":"2024-04-25T01:08:24.476673+0000","last_active":"2024-04-25T01:08:29.511101+0000","last_peered":"2024-04-25T01:08:29.511101+0000","last_clean":"2024-04-25T01:08:29.511101+0000","last_became_active":"2024-04-25T01:08:24.471388+0000","last_became_peered":"2024-04-25T01:08:24.471388+0000","last_unstale":"2024-04-25T01:08:29.511101+0000","last_undegraded":"2024-04-25T01:08:29.511101+0000","last_fullsized":"2024-04-25T01:08:29.511101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:36.539667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509254+0000","last_change":"2024-04-25T01:08:28.539720+0000","last_active":"2024-04-25T01:08:29.509254+0000","last_peered":"2024-04-25T01:08:29.509254+0000","last_clean":"2024-04-25T01:08:29.509254+0000","last_became_active":"2024-04-25T01:08:28.538885+0000","last_became_peered":"2024-04-25T01:08:28.538885+0000","last_unstale":"2024-04-25T01:08:29.509254+0000","last_undegraded":"2024-04-25T01:08:29.509254+0000","last_fullsized":"2024-04-25T01:08:29.509254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:51.273909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824379+0000","last_change":"2024-04-25T01:08:24.499102+0000","last_active":"2024-04-25T01:08:29.824379+0000","last_peered":"2024-04-25T01:08:29.824379+0000","last_clean":"2024-04-25T01:08:29.824379+0000","last_became_active":"2024-04-25T01:08:24.498282+0000","last_became_peered":"2024-04-25T01:08:24.498282+0000","last_unstale":"2024-04-25T01:08:29.824379+0000","last_undegraded":"2024-04-25T01:08:29.824379+0000","last_fullsized":"2024-04-25T01:08:29.824379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824785+0000","last_change":"2024-04-25T01:08:24.477024+0000","last_active":"2024-04-25T01:08:29.824785+0000","last_peered":"2024-04-25T01:08:29.824785+0000","last_clean":"2024-04-25T01:08:29.824785+0000","last_became_active":"2024-04-25T01:08:24.476723+0000","last_became_peered":"2024-04-25T01:08:24.476723+0000","last_unstale":"2024-04-25T01:08:29.824785+0000","last_undegraded":"2024-04-25T01:08:29.824785+0000","last_fullsized":"2024-04-25T01:08:29.824785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825769+0000","last_change":"2024-04-25T01:08:24.492781+0000","last_active":"2024-04-25T01:08:29.825769+0000","last_peered":"2024-04-25T01:08:29.825769+0000","last_clean":"2024-04-25T01:08:29.825769+0000","last_became_active":"2024-04-25T01:08:24.492041+0000","last_became_peered":"2024-04-25T01:08:24.492041+0000","last_unstale":"2024-04-25T01:08:29.825769+0000","last_undegraded":"2024-04-25T01:08:29.825769+0000","last_fullsized":"2024-04-25T01:08:29.825769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:35.171404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827611+0000","last_change":"2024-04-25T01:08:24.493915+0000","last_active":"2024-04-25T01:08:29.827611+0000","last_peered":"2024-04-25T01:08:29.827611+0000","last_clean":"2024-04-25T01:08:29.827611+0000","last_became_active":"2024-04-25T01:08:24.493620+0000","last_became_peered":"2024-04-25T01:08:24.493620+0000","last_unstale":"2024-04-25T01:08:29.827611+0000","last_undegraded":"2024-04-25T01:08:29.827611+0000","last_fullsized":"2024-04-25T01:08:29.827611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:42.788724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509565+0000","last_change":"2024-04-25T01:08:24.491720+0000","last_active":"2024-04-25T01:08:29.509565+0000","last_peered":"2024-04-25T01:08:29.509565+0000","last_clean":"2024-04-25T01:08:29.509565+0000","last_became_active":"2024-04-25T01:08:24.491474+0000","last_became_peered":"2024-04-25T01:08:24.491474+0000","last_unstale":"2024-04-25T01:08:29.509565+0000","last_undegraded":"2024-04-25T01:08:29.509565+0000","last_fullsized":"2024-04-25T01:08:29.509565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828668+0000","last_change":"2024-04-25T01:08:24.472108+0000","last_active":"2024-04-25T01:08:29.828668+0000","last_peered":"2024-04-25T01:08:29.828668+0000","last_clean":"2024-04-25T01:08:29.828668+0000","last_became_active":"2024-04-25T01:08:24.471688+0000","last_became_peered":"2024-04-25T01:08:24.471688+0000","last_unstale":"2024-04-25T01:08:29.828668+0000","last_undegraded":"2024-04-25T01:08:29.828668+0000","last_fullsized":"2024-04-25T01:08:29.828668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829144+0000","last_change":"2024-04-25T01:08:24.515413+0000","last_active":"2024-04-25T01:08:29.829144+0000","last_peered":"2024-04-25T01:08:29.829144+0000","last_clean":"2024-04-25T01:08:29.829144+0000","last_became_active":"2024-04-25T01:08:24.514573+0000","last_became_peered":"2024-04-25T01:08:24.514573+0000","last_unstale":"2024-04-25T01:08:29.829144+0000","last_undegraded":"2024-04-25T01:08:29.829144+0000","last_fullsized":"2024-04-25T01:08:29.829144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510173+0000","last_change":"2024-04-25T01:08:24.487417+0000","last_active":"2024-04-25T01:08:29.510173+0000","last_peered":"2024-04-25T01:08:29.510173+0000","last_clean":"2024-04-25T01:08:29.510173+0000","last_became_active":"2024-04-25T01:08:24.487135+0000","last_became_peered":"2024-04-25T01:08:24.487135+0000","last_unstale":"2024-04-25T01:08:29.510173+0000","last_undegraded":"2024-04-25T01:08:29.510173+0000","last_fullsized":"2024-04-25T01:08:29.510173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825289+0000","last_change":"2024-04-25T01:08:24.484688+0000","last_active":"2024-04-25T01:08:29.825289+0000","last_peered":"2024-04-25T01:08:29.825289+0000","last_clean":"2024-04-25T01:08:29.825289+0000","last_became_active":"2024-04-25T01:08:24.483926+0000","last_became_peered":"2024-04-25T01:08:24.483926+0000","last_unstale":"2024-04-25T01:08:29.825289+0000","last_undegraded":"2024-04-25T01:08:29.825289+0000","last_fullsized":"2024-04-25T01:08:29.825289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508350+0000","last_change":"2024-04-25T01:08:24.487479+0000","last_active":"2024-04-25T01:08:29.508350+0000","last_peered":"2024-04-25T01:08:29.508350+0000","last_clean":"2024-04-25T01:08:29.508350+0000","last_became_active":"2024-04-25T01:08:24.487177+0000","last_became_peered":"2024-04-25T01:08:24.487177+0000","last_unstale":"2024-04-25T01:08:29.508350+0000","last_undegraded":"2024-04-25T01:08:29.508350+0000","last_fullsized":"2024-04-25T01:08:29.508350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825549+0000","last_change":"2024-04-25T01:08:24.516077+0000","last_active":"2024-04-25T01:08:29.825549+0000","last_peered":"2024-04-25T01:08:29.825549+0000","last_clean":"2024-04-25T01:08:29.825549+0000","last_became_active":"2024-04-25T01:08:24.515560+0000","last_became_peered":"2024-04-25T01:08:24.515560+0000","last_unstale":"2024-04-25T01:08:29.825549+0000","last_undegraded":"2024-04-25T01:08:29.825549+0000","last_fullsized":"2024-04-25T01:08:29.825549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:30.731050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827066+0000","last_change":"2024-04-25T01:08:24.497188+0000","last_active":"2024-04-25T01:08:29.827066+0000","last_peered":"2024-04-25T01:08:29.827066+0000","last_clean":"2024-04-25T01:08:29.827066+0000","last_became_active":"2024-04-25T01:08:24.494236+0000","last_became_peered":"2024-04-25T01:08:24.494236+0000","last_unstale":"2024-04-25T01:08:29.827066+0000","last_undegraded":"2024-04-25T01:08:29.827066+0000","last_fullsized":"2024-04-25T01:08:29.827066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:27.907903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825891+0000","last_change":"2024-04-25T01:08:24.497623+0000","last_active":"2024-04-25T01:08:29.825891+0000","last_peered":"2024-04-25T01:08:29.825891+0000","last_clean":"2024-04-25T01:08:29.825891+0000","last_became_active":"2024-04-25T01:08:24.488300+0000","last_became_peered":"2024-04-25T01:08:24.488300+0000","last_unstale":"2024-04-25T01:08:29.825891+0000","last_undegraded":"2024-04-25T01:08:29.825891+0000","last_fullsized":"2024-04-25T01:08:29.825891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826575+0000","last_change":"2024-04-25T01:08:24.525066+0000","last_active":"2024-04-25T01:08:29.826575+0000","last_peered":"2024-04-25T01:08:29.826575+0000","last_clean":"2024-04-25T01:08:29.826575+0000","last_became_active":"2024-04-25T01:08:24.524022+0000","last_became_peered":"2024-04-25T01:08:24.524022+0000","last_unstale":"2024-04-25T01:08:29.826575+0000","last_undegraded":"2024-04-25T01:08:29.826575+0000","last_fullsized":"2024-04-25T01:08:29.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:33.317294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828523+0000","last_change":"2024-04-25T01:08:24.474988+0000","last_active":"2024-04-25T01:08:29.828523+0000","last_peered":"2024-04-25T01:08:29.828523+0000","last_clean":"2024-04-25T01:08:29.828523+0000","last_became_active":"2024-04-25T01:08:24.472840+0000","last_became_peered":"2024-04-25T01:08:24.472840+0000","last_unstale":"2024-04-25T01:08:29.828523+0000","last_undegraded":"2024-04-25T01:08:29.828523+0000","last_fullsized":"2024-04-25T01:08:29.828523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827354+0000","last_change":"2024-04-25T01:08:24.504414+0000","last_active":"2024-04-25T01:08:29.827354+0000","last_peered":"2024-04-25T01:08:29.827354+0000","last_clean":"2024-04-25T01:08:29.827354+0000","last_became_active":"2024-04-25T01:08:24.503622+0000","last_became_peered":"2024-04-25T01:08:24.503622+0000","last_unstale":"2024-04-25T01:08:29.827354+0000","last_undegraded":"2024-04-25T01:08:29.827354+0000","last_fullsized":"2024-04-25T01:08:29.827354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:01.970030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825048+0000","last_change":"2024-04-25T01:08:24.496532+0000","last_active":"2024-04-25T01:08:29.825048+0000","last_peered":"2024-04-25T01:08:29.825048+0000","last_clean":"2024-04-25T01:08:29.825048+0000","last_became_active":"2024-04-25T01:08:24.495812+0000","last_became_peered":"2024-04-25T01:08:24.495812+0000","last_unstale":"2024-04-25T01:08:29.825048+0000","last_undegraded":"2024-04-25T01:08:29.825048+0000","last_fullsized":"2024-04-25T01:08:29.825048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828165+0000","last_change":"2024-04-25T01:08:24.461285+0000","last_active":"2024-04-25T01:08:29.828165+0000","last_peered":"2024-04-25T01:08:29.828165+0000","last_clean":"2024-04-25T01:08:29.828165+0000","last_became_active":"2024-04-25T01:08:24.460929+0000","last_became_peered":"2024-04-25T01:08:24.460929+0000","last_unstale":"2024-04-25T01:08:29.828165+0000","last_undegraded":"2024-04-25T01:08:29.828165+0000","last_fullsized":"2024-04-25T01:08:29.828165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827126+0000","last_change":"2024-04-25T01:08:24.493618+0000","last_active":"2024-04-25T01:08:29.827126+0000","last_peered":"2024-04-25T01:08:29.827126+0000","last_clean":"2024-04-25T01:08:29.827126+0000","last_became_active":"2024-04-25T01:08:24.489439+0000","last_became_peered":"2024-04-25T01:08:24.489439+0000","last_unstale":"2024-04-25T01:08:29.827126+0000","last_undegraded":"2024-04-25T01:08:29.827126+0000","last_fullsized":"2024-04-25T01:08:29.827126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:43.940192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508267+0000","last_change":"2024-04-25T01:08:24.478846+0000","last_active":"2024-04-25T01:08:29.508267+0000","last_peered":"2024-04-25T01:08:29.508267+0000","last_clean":"2024-04-25T01:08:29.508267+0000","last_became_active":"2024-04-25T01:08:24.478580+0000","last_became_peered":"2024-04-25T01:08:24.478580+0000","last_unstale":"2024-04-25T01:08:29.508267+0000","last_undegraded":"2024-04-25T01:08:29.508267+0000","last_fullsized":"2024-04-25T01:08:29.508267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511438+0000","last_change":"2024-04-25T01:08:24.481980+0000","last_active":"2024-04-25T01:08:29.511438+0000","last_peered":"2024-04-25T01:08:29.511438+0000","last_clean":"2024-04-25T01:08:29.511438+0000","last_became_active":"2024-04-25T01:08:24.480214+0000","last_became_peered":"2024-04-25T01:08:24.480214+0000","last_unstale":"2024-04-25T01:08:29.511438+0000","last_undegraded":"2024-04-25T01:08:29.511438+0000","last_fullsized":"2024-04-25T01:08:29.511438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826879+0000","last_change":"2024-04-25T01:08:24.480212+0000","last_active":"2024-04-25T01:08:29.826879+0000","last_peered":"2024-04-25T01:08:29.826879+0000","last_clean":"2024-04-25T01:08:29.826879+0000","last_became_active":"2024-04-25T01:08:24.477781+0000","last_became_peered":"2024-04-25T01:08:24.477781+0000","last_unstale":"2024-04-25T01:08:29.826879+0000","last_undegraded":"2024-04-25T01:08:29.826879+0000","last_fullsized":"2024-04-25T01:08:29.826879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829093+0000","last_change":"2024-04-25T01:08:24.480926+0000","last_active":"2024-04-25T01:08:29.829093+0000","last_peered":"2024-04-25T01:08:29.829093+0000","last_clean":"2024-04-25T01:08:29.829093+0000","last_became_active":"2024-04-25T01:08:24.477787+0000","last_became_peered":"2024-04-25T01:08:24.477787+0000","last_unstale":"2024-04-25T01:08:29.829093+0000","last_undegraded":"2024-04-25T01:08:29.829093+0000","last_fullsized":"2024-04-25T01:08:29.829093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827821+0000","last_change":"2024-04-25T01:08:24.525497+0000","last_active":"2024-04-25T01:08:29.827821+0000","last_peered":"2024-04-25T01:08:29.827821+0000","last_clean":"2024-04-25T01:08:29.827821+0000","last_became_active":"2024-04-25T01:08:24.524512+0000","last_became_peered":"2024-04-25T01:08:24.524512+0000","last_unstale":"2024-04-25T01:08:29.827821+0000","last_undegraded":"2024-04-25T01:08:29.827821+0000","last_fullsized":"2024-04-25T01:08:29.827821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:35.642716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507922+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.507922+0000","last_peered":"2024-04-25T01:08:29.507922+0000","last_clean":"2024-04-25T01:08:29.507922+0000","last_became_active":"2024-04-25T01:08:24.486493+0000","last_became_peered":"2024-04-25T01:08:24.486493+0000","last_unstale":"2024-04-25T01:08:29.507922+0000","last_undegraded":"2024-04-25T01:08:29.507922+0000","last_fullsized":"2024-04-25T01:08:29.507922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828202+0000","last_change":"2024-04-25T01:08:24.510776+0000","last_active":"2024-04-25T01:08:29.828202+0000","last_peered":"2024-04-25T01:08:29.828202+0000","last_clean":"2024-04-25T01:08:29.828202+0000","last_became_active":"2024-04-25T01:08:24.509980+0000","last_became_peered":"2024-04-25T01:08:24.509980+0000","last_unstale":"2024-04-25T01:08:29.828202+0000","last_undegraded":"2024-04-25T01:08:29.828202+0000","last_fullsized":"2024-04-25T01:08:29.828202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825321+0000","last_change":"2024-04-25T01:08:24.497109+0000","last_active":"2024-04-25T01:08:29.825321+0000","last_peered":"2024-04-25T01:08:29.825321+0000","last_clean":"2024-04-25T01:08:29.825321+0000","last_became_active":"2024-04-25T01:08:24.496213+0000","last_became_peered":"2024-04-25T01:08:24.496213+0000","last_unstale":"2024-04-25T01:08:29.825321+0000","last_undegraded":"2024-04-25T01:08:29.825321+0000","last_fullsized":"2024-04-25T01:08:29.825321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827021+0000","last_change":"2024-04-25T01:08:24.524338+0000","last_active":"2024-04-25T01:08:29.827021+0000","last_peered":"2024-04-25T01:08:29.827021+0000","last_clean":"2024-04-25T01:08:29.827021+0000","last_became_active":"2024-04-25T01:08:24.523936+0000","last_became_peered":"2024-04-25T01:08:24.523936+0000","last_unstale":"2024-04-25T01:08:29.827021+0000","last_undegraded":"2024-04-25T01:08:29.827021+0000","last_fullsized":"2024-04-25T01:08:29.827021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:50.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826302+0000","last_change":"2024-04-25T01:08:24.491543+0000","last_active":"2024-04-25T01:08:29.826302+0000","last_peered":"2024-04-25T01:08:29.826302+0000","last_clean":"2024-04-25T01:08:29.826302+0000","last_became_active":"2024-04-25T01:08:24.490866+0000","last_became_peered":"2024-04-25T01:08:24.490866+0000","last_unstale":"2024-04-25T01:08:29.826302+0000","last_undegraded":"2024-04-25T01:08:29.826302+0000","last_fullsized":"2024-04-25T01:08:29.826302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:38.093629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828785+0000","last_change":"2024-04-25T01:08:24.459335+0000","last_active":"2024-04-25T01:08:29.828785+0000","last_peered":"2024-04-25T01:08:29.828785+0000","last_clean":"2024-04-25T01:08:29.828785+0000","last_became_active":"2024-04-25T01:08:24.458795+0000","last_became_peered":"2024-04-25T01:08:24.458795+0000","last_unstale":"2024-04-25T01:08:29.828785+0000","last_undegraded":"2024-04-25T01:08:29.828785+0000","last_fullsized":"2024-04-25T01:08:29.828785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:10.682483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829652+0000","last_change":"2024-04-25T01:08:24.465465+0000","last_active":"2024-04-25T01:08:29.829652+0000","last_peered":"2024-04-25T01:08:29.829652+0000","last_clean":"2024-04-25T01:08:29.829652+0000","last_became_active":"2024-04-25T01:08:24.464433+0000","last_became_peered":"2024-04-25T01:08:24.464433+0000","last_unstale":"2024-04-25T01:08:29.829652+0000","last_undegraded":"2024-04-25T01:08:29.829652+0000","last_fullsized":"2024-04-25T01:08:29.829652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827333+0000","last_change":"2024-04-25T01:08:24.525387+0000","last_active":"2024-04-25T01:08:29.827333+0000","last_peered":"2024-04-25T01:08:29.827333+0000","last_clean":"2024-04-25T01:08:29.827333+0000","last_became_active":"2024-04-25T01:08:24.524324+0000","last_became_peered":"2024-04-25T01:08:24.524324+0000","last_unstale":"2024-04-25T01:08:29.827333+0000","last_undegraded":"2024-04-25T01:08:29.827333+0000","last_fullsized":"2024-04-25T01:08:29.827333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:49.276483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":102463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":102553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":693469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:08:42.591 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T01:08:42.591 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T01:08:42.592 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T01:08:42.592 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T01:08:42.592 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T01:08:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a2669640 1 Processor -- start 2024-04-25T01:08:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a2669640 1 -- start start 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a2669640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff09c0d03e0 0x7ff09c0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a2669640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 0x7ff09c0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a2669640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff09c0d10b0 con 0x7ff09c0d48d0 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a2669640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff09c0d1de0 con 0x7ff09c0d03e0 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a2669640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c0d1390 con 0x7ff09c15e470 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a0c06640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff09c0d48d0 0x7ff09c0d27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33852/0 (socket says 172.21.15.33:33852) 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a0c06640 1 -- 172.21.15.33:0/2606867112 learned_addr learned my addr 172.21.15.33:0/2606867112 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 --2- 172.21.15.33:0/2606867112 >> v2:172.21.15.138:3300/0 conn(0x7ff09c0d03e0 0x7ff09c0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09b7fe640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 0x7ff09c0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1024601466 0 0) 0x7ff09c0d10b0 con 0x7ff09c0d48d0 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff088003530 con 0x7ff09c0d48d0 2024-04-25T01:08:42.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065948710 0 0) 0x7ff088003530 con 0x7ff09c0d48d0 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 msgr2=0x7ff09c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 0x7ff09c0d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 >> v2:172.21.15.138:3300/0 conn(0x7ff09c0d03e0 msgr2=0x7ff09c0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 --2- 172.21.15.33:0/2606867112 >> v2:172.21.15.138:3300/0 conn(0x7ff09c0d03e0 0x7ff09c0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0d1670 con 0x7ff09c0d48d0 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7ff094001680 con 0x7ff09c0d48d0 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 --2- 172.21.15.33:0/2606867112 >> v2:172.21.15.138:3300/0 conn(0x7ff09c0d03e0 0x7ff09c0d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09b7fe640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 0x7ff09c0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 >> v1:172.21.15.33:6789/0 conn(0x7ff09c0d48d0 legacy=0x7ff09c0d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0880039c0 0x7ff088003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff088004810 0x7ff088004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 0x7ff0880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0d1de0 con 0x7ff0880039c0 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c0d1390 con 0x7ff088004810 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff08800b9d0 con 0x7ff09c15e470 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0880039c0 0x7ff088003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09b7fe640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff088004810 0x7ff088004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff0a0c06640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 0x7ff0880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 -- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 msgr2=0x7ff0880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff09c15e470 0x7ff0880051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 -- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff088004810 msgr2=0x7ff088004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff088004810 0x7ff088004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.761+0000 7ff09bfff640 1 -- 172.21.15.33:0/2606867112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff088003530 con 0x7ff0880039c0 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff09bfff640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0880039c0 0x7ff088003de0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009cb0 tx=0x7ff08c01cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97696824b4f14a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0b2070 con 0x7ff0880039c0 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff08c0a56a0 con 0x7ff0880039c0 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff09affd640 1 -- 172.21.15.33:0/2606867112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff08c009880 con 0x7ff0880039c0 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff0a2669640 1 -- 172.21.15.33:0/2606867112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0880039c0 msgr2=0x7ff088003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff0a2669640 1 --2- 172.21.15.33:0/2606867112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0880039c0 0x7ff088003de0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009cb0 tx=0x7ff08c01cbd0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff0a0c06640 1 -- 172.21.15.33:0/2606867112 reap_dead start 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff0a2669640 1 -- 172.21.15.33:0/2606867112 shutdown_connections 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff0a2669640 1 -- 172.21.15.33:0/2606867112 >> 172.21.15.33:0/2606867112 conn(0x7ff09c0d3db0 msgr2=0x7ff09c0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff0a2669640 1 -- 172.21.15.33:0/2606867112 shutdown_connections 2024-04-25T01:08:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.765+0000 7ff0a2669640 1 -- 172.21.15.33:0/2606867112 wait complete. 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 Processor -- start 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 -- start start 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0880039c0 0x7ff09c0cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c15fa30 0x7ff09c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff09c0ccd30 0x7ff09c14c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c168f30 con 0x7ff09c0ccd30 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff09c168db0 con 0x7ff09c15fa30 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff09c1690b0 con 0x7ff0880039c0 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff09c0ccd30 0x7ff09c14c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff09c0ccd30 0x7ff09c14c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49874/0 (socket says 172.21.15.33:49874) 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 -- 172.21.15.33:0/1226610722 learned_addr learned my addr 172.21.15.33:0/1226610722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff09bfff640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0880039c0 0x7ff09c0cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff09b7fe640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c15fa30 0x7ff09c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 -- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0880039c0 msgr2=0x7ff09c0cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0880039c0 0x7ff09c0cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 -- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c15fa30 msgr2=0x7ff09c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c15fa30 0x7ff09c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 -- 172.21.15.33:0/1226610722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff09c0cba30 con 0x7ff09c0ccd30 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a0c06640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff09c0ccd30 0x7ff09c14c550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff094000f70 tx=0x7ff09400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff098ff9640 1 -- 172.21.15.33:0/1226610722 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094007230 con 0x7ff09c0ccd30 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff098ff9640 1 -- 172.21.15.33:0/1226610722 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0940073f0 con 0x7ff09c0ccd30 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff098ff9640 1 -- 172.21.15.33:0/1226610722 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0940047f0 con 0x7ff09c0ccd30 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff09c0cbd10 con 0x7ff09c0ccd30 2024-04-25T01:08:42.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff09c0cc2d0 con 0x7ff09c0ccd30 2024-04-25T01:08:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff09c069040 con 0x7ff09c0ccd30 2024-04-25T01:08:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff098ff9640 1 -- 172.21.15.33:0/1226610722 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff09401b060 con 0x7ff09c0ccd30 2024-04-25T01:08:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff098ff9640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff07c0678c0 0x7ff07c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.769+0000 7ff098ff9640 1 -- 172.21.15.33:0/1226610722 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff0940858b0 con 0x7ff09c0ccd30 2024-04-25T01:08:42.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.773+0000 7ff09bfff640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff07c0678c0 0x7ff07c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:42.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.773+0000 7ff09bfff640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff07c0678c0 0x7ff07c069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0af000 tx=0x7ff08c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:42.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.777+0000 7ff098ff9640 1 -- 172.21.15.33:0/1226610722 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff0940560b0 con 0x7ff09c0ccd30 2024-04-25T01:08:42.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.969+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7ff09c0cc4a0 con 0x7ff09c0ccd30 2024-04-25T01:08:42.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.977+0000 7ff098ff9640 1 -- 172.21.15.33:0/1226610722 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166891 (secure 0 0 0) 0x7ff0940559b0 con 0x7ff09c0ccd30 2024-04-25T01:08:42.972 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff07c0678c0 msgr2=0x7ff07c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff07c0678c0 0x7ff07c069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff08c0af000 tx=0x7ff08c0046d0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff09c0ccd30 msgr2=0x7ff09c14c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff09c0ccd30 0x7ff09c14c550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff094000f70 tx=0x7ff09400dc00 comp rx=0 tx=0).stop 2024-04-25T01:08:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 shutdown_connections 2024-04-25T01:08:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff07c0678c0 0x7ff07c069d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff09c0ccd30 0x7ff09c14c550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff09c15fa30 0x7ff09c0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 --2- 172.21.15.33:0/1226610722 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff0880039c0 0x7ff09c0cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 >> 172.21.15.33:0/1226610722 conn(0x7ff09c0d3db0 msgr2=0x7ff09c160340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 shutdown_connections 2024-04-25T01:08:42.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:42.981+0000 7ff0a2669640 1 -- 172.21.15.33:0/1226610722 wait complete. 2024-04-25T01:08:42.988 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":2108469693},{"type":"v1","addr":"172.21.15.138:6833","nonce":2108469693}]},"active_addr":"172.21.15.138:6833/2108469693","active_change":"2024-04-25T01:08:11.074085+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":481125859}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2345762231}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1614085320}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":37710293}]}]} 2024-04-25T01:08:42.991 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T01:08:42.991 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T01:08:42.991 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:43.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca34c61640 1 Processor -- start 2024-04-25T01:08:43.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca34c61640 1 -- start start 2024-04-25T01:08:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca34c61640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca30145470 0x7fca3015c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca34c61640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3015f9a0 con 0x7fca300b5530 2024-04-25T01:08:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca34c61640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3015fc80 con 0x7fca300cf0e0 2024-04-25T01:08:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca34c61640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fca3015f750 con 0x7fca30145470 2024-04-25T01:08:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca2e59c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fca30145470 0x7fca3015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca2ed9d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fca300b5530 0x7fca300cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33858/0 (socket says 172.21.15.33:33858) 2024-04-25T01:08:43.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca2ed9d640 1 -- 172.21.15.33:0/3357689501 learned_addr learned my addr 172.21.15.33:0/3357689501 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca2e59c640 1 -- 172.21.15.33:0/3357689501 >> v1:172.21.15.138:6789/0 conn(0x7fca300cf0e0 legacy=0x7fca300b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca2e59c640 1 -- 172.21.15.33:0/3357689501 >> v1:172.21.15.33:6789/0 conn(0x7fca300b5530 legacy=0x7fca300cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca2d59a640 1 -- 172.21.15.33:0/3357689501 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3153267531 0 0) 0x7fca3015f9a0 con 0x7fca300b5530 2024-04-25T01:08:43.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.117+0000 7fca2e59c640 1 -- 172.21.15.33:0/3357689501 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3015ff60 con 0x7fca30145470 2024-04-25T01:08:43.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca2e59c640 1 --2- 172.21.15.33:0/3357689501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca3015f8d0 tx=0x7fca2001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83d46aa1ce33be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca2d59a640 1 -- 172.21.15.33:0/3357689501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca200ab620 con 0x7fca30145470 2024-04-25T01:08:43.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca2d59a640 1 -- 172.21.15.33:0/3357689501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca200b0070 con 0x7fca30145470 2024-04-25T01:08:43.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca2d59a640 1 -- 172.21.15.33:0/3357689501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca200ab9e0 con 0x7fca30145470 2024-04-25T01:08:43.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 -- 172.21.15.33:0/3357689501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 msgr2=0x7fca3015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 --2- 172.21.15.33:0/3357689501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c160 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca3015f8d0 tx=0x7fca2001c460 comp rx=0 tx=0).stop 2024-04-25T01:08:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 -- 172.21.15.33:0/3357689501 shutdown_connections 2024-04-25T01:08:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 --2- 172.21.15.33:0/3357689501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c160 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 -- 172.21.15.33:0/3357689501 >> 172.21.15.33:0/3357689501 conn(0x7fca300b1810 msgr2=0x7fca300b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 -- 172.21.15.33:0/3357689501 shutdown_connections 2024-04-25T01:08:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 -- 172.21.15.33:0/3357689501 wait complete. 2024-04-25T01:08:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 Processor -- start 2024-04-25T01:08:43.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.121+0000 7fca34c61640 1 -- start start 2024-04-25T01:08:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca300b5530 0x7fca3015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca300cf0e0 0x7fca30134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca30169c80 con 0x7fca30145470 2024-04-25T01:08:43.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca30169b00 con 0x7fca300cf0e0 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fca30169e00 con 0x7fca300b5530 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49884/0 (socket says 172.21.15.33:49884) 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 -- 172.21.15.33:0/1731396542 learned_addr learned my addr 172.21.15.33:0/1731396542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2e59c640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca300b5530 0x7fca3015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 -- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca300b5530 msgr2=0x7fca3015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca300b5530 0x7fca3015b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 -- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca300cf0e0 msgr2=0x7fca30134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca300cf0e0 0x7fca30134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 -- 172.21.15.33:0/1731396542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3015b600 con 0x7fca30145470 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca2ed9d640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c700 tx=0x7fca2800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca26ffd640 1 -- 172.21.15.33:0/1731396542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28018070 con 0x7fca30145470 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca26ffd640 1 -- 172.21.15.33:0/1731396542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2800ce10 con 0x7fca30145470 2024-04-25T01:08:43.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca30144810 con 0x7fca30145470 2024-04-25T01:08:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca26ffd640 1 -- 172.21.15.33:0/1731396542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2800a860 con 0x7fca30145470 2024-04-25T01:08:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca30143f70 con 0x7fca30145470 2024-04-25T01:08:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca30069040 con 0x7fca30145470 2024-04-25T01:08:43.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.125+0000 7fca26ffd640 1 -- 172.21.15.33:0/1731396542 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fca2801b070 con 0x7fca30145470 2024-04-25T01:08:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.129+0000 7fca26ffd640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678c0 0x7fca10069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.129+0000 7fca26ffd640 1 -- 172.21.15.33:0/1731396542 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fca28085850 con 0x7fca30145470 2024-04-25T01:08:43.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.129+0000 7fca2e59c640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678c0 0x7fca10069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.129+0000 7fca2e59c640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678c0 0x7fca10069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fca200af000 tx=0x7fca20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.129+0000 7fca26ffd640 1 -- 172.21.15.33:0/1731396542 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fca280560c0 con 0x7fca30145470 2024-04-25T01:08:43.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.297+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca30071420 con 0x7fca30145470 2024-04-25T01:08:43.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca26ffd640 1 -- 172.21.15.33:0/1731396542 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fca280559c0 con 0x7fca30145470 2024-04-25T01:08:43.293 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:43.293 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678c0 msgr2=0x7fca10069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678c0 0x7fca10069d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fca200af000 tx=0x7fca20008040 comp rx=0 tx=0).stop 2024-04-25T01:08:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 msgr2=0x7fca3015c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c700 tx=0x7fca2800cbc0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 shutdown_connections 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fca100678c0 0x7fca10069d80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca30145470 0x7fca3015c650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca300cf0e0 0x7fca30134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 --2- 172.21.15.33:0/1731396542 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fca300b5530 0x7fca3015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 >> 172.21.15.33:0/1731396542 conn(0x7fca300b1810 msgr2=0x7fca3013d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 shutdown_connections 2024-04-25T01:08:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.301+0000 7fca34c61640 1 -- 172.21.15.33:0/1731396542 wait complete. 2024-04-25T01:08:43.313 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T01:08:43.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd9ab57640 1 Processor -- start 2024-04-25T01:08:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd9ab57640 1 -- start start 2024-04-25T01:08:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd9ab57640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940d4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd9ab57640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd940d2430 0x7fdd940d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd9ab57640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd940b6d40 con 0x7fdd940d0da0 2024-04-25T01:08:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd9ab57640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fdd940d16e0 con 0x7fdd940d2430 2024-04-25T01:08:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd9ab57640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9415e4c0 con 0x7fdd9415e0e0 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd990f4640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fdd940d0da0 0x7fdd940ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33872/0 (socket says 172.21.15.33:33872) 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd990f4640 1 -- 172.21.15.33:0/3060105574 learned_addr learned my addr 172.21.15.33:0/3060105574 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.441+0000 7fdd93fff640 1 --2- 172.21.15.33:0/3060105574 >> v2:172.21.15.33:3300/0 conn(0x7fdd940d2430 0x7fdd940d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd988f3640 1 --2- 172.21.15.33:0/3060105574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940d4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd93fff640 1 -- 172.21.15.33:0/3060105574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 msgr2=0x7fdd940d4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd93fff640 1 --2- 172.21.15.33:0/3060105574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940d4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd93fff640 1 -- 172.21.15.33:0/3060105574 >> v1:172.21.15.33:6789/0 conn(0x7fdd940d0da0 legacy=0x7fdd940ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd937fe640 1 -- 172.21.15.33:0/3060105574 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1055777148 0 0) 0x7fdd940b6d40 con 0x7fdd940d0da0 2024-04-25T01:08:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd93fff640 1 -- 172.21.15.33:0/3060105574 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd940d29d0 con 0x7fdd940d2430 2024-04-25T01:08:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd93fff640 1 --2- 172.21.15.33:0/3060105574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d2430 0x7fdd940d0860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd880097c0 tx=0x7fdd8801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e69370361927b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd937fe640 1 -- 172.21.15.33:0/3060105574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880b2070 con 0x7fdd940d2430 2024-04-25T01:08:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd937fe640 1 -- 172.21.15.33:0/3060105574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880ad640 con 0x7fdd940d2430 2024-04-25T01:08:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd937fe640 1 -- 172.21.15.33:0/3060105574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880ada10 con 0x7fdd940d2430 2024-04-25T01:08:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 -- 172.21.15.33:0/3060105574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d2430 msgr2=0x7fdd940d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/3060105574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d2430 0x7fdd940d0860 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd880097c0 tx=0x7fdd8801c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 -- 172.21.15.33:0/3060105574 shutdown_connections 2024-04-25T01:08:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/3060105574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d2430 0x7fdd940d0860 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/3060105574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940d4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 -- 172.21.15.33:0/3060105574 >> 172.21.15.33:0/3060105574 conn(0x7fdd940d3e50 msgr2=0x7fdd940d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 -- 172.21.15.33:0/3060105574 shutdown_connections 2024-04-25T01:08:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.445+0000 7fdd9ab57640 1 -- 172.21.15.33:0/3060105574 wait complete. 2024-04-25T01:08:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 Processor -- start 2024-04-25T01:08:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 -- start start 2024-04-25T01:08:43.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d0da0 0x7fdd94156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd940d2430 0x7fdd940caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9416ad90 con 0x7fdd940d0da0 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9416ac10 con 0x7fdd940d2430 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd9416af10 con 0x7fdd9415e0e0 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d0da0 0x7fdd94156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d0da0 0x7fdd94156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49898/0 (socket says 172.21.15.33:49898) 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 -- 172.21.15.33:0/270203337 learned_addr learned my addr 172.21.15.33:0/270203337 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd93fff640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd940d2430 0x7fdd940caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 -- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 msgr2=0x7fdd940cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd990f4640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940cb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 -- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd940d2430 msgr2=0x7fdd940caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd940d2430 0x7fdd940caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 -- 172.21.15.33:0/270203337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd940cc210 con 0x7fdd940d0da0 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd93fff640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd940d2430 0x7fdd940caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd988f3640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d0da0 0x7fdd94156dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e8a0 tx=0x7fdd8400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd917fa640 1 -- 172.21.15.33:0/270203337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84017070 con 0x7fdd940d0da0 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd917fa640 1 -- 172.21.15.33:0/270203337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8400c940 con 0x7fdd940d0da0 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd940cc4f0 con 0x7fdd940d0da0 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd917fa640 1 -- 172.21.15.33:0/270203337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400ccf0 con 0x7fdd940d0da0 2024-04-25T01:08:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd940ccab0 con 0x7fdd940d0da0 2024-04-25T01:08:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.449+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd94069040 con 0x7fdd940d0da0 2024-04-25T01:08:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.453+0000 7fdd917fa640 1 -- 172.21.15.33:0/270203337 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdd8401a070 con 0x7fdd940d0da0 2024-04-25T01:08:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.453+0000 7fdd917fa640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd800678c0 0x7fdd80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.453+0000 7fdd917fa640 1 -- 172.21.15.33:0/270203337 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdd84086260 con 0x7fdd940d0da0 2024-04-25T01:08:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.453+0000 7fdd93fff640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd800678c0 0x7fdd80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.453+0000 7fdd93fff640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd800678c0 0x7fdd80069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd880af000 tx=0x7fdd88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.457+0000 7fdd917fa640 1 -- 172.21.15.33:0/270203337 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdd84056a60 con 0x7fdd940d0da0 2024-04-25T01:08:43.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.621+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd94073cd0 con 0x7fdd940d0da0 2024-04-25T01:08:43.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.621+0000 7fdd917fa640 1 -- 172.21.15.33:0/270203337 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fdd84056360 con 0x7fdd940d0da0 2024-04-25T01:08:43.616 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:43.616 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:43.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd800678c0 msgr2=0x7fdd80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd800678c0 0x7fdd80069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd880af000 tx=0x7fdd88008040 comp rx=0 tx=0).stop 2024-04-25T01:08:43.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d0da0 msgr2=0x7fdd94156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d0da0 0x7fdd94156dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400e8a0 tx=0x7fdd8400eca0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 shutdown_connections 2024-04-25T01:08:43.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd800678c0 0x7fdd80069d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd9415e0e0 0x7fdd940cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd940d2430 0x7fdd940caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 --2- 172.21.15.33:0/270203337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd940d0da0 0x7fdd94156dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 >> 172.21.15.33:0/270203337 conn(0x7fdd940d3e50 msgr2=0x7fdd940d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 shutdown_connections 2024-04-25T01:08:43.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.625+0000 7fdd9ab57640 1 -- 172.21.15.33:0/270203337 wait complete. 2024-04-25T01:08:43.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T01:08:43.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T01:08:43.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T01:08:43.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T01:08:43.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T01:08:43.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T01:08:43.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T01:08:43.638 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T01:08:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 Processor -- start 2024-04-25T01:08:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 -- start start 2024-04-25T01:08:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f3395232640 1 Processor -- start 2024-04-25T01:08:43.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f3395232640 1 -- start start 2024-04-25T01:08:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb5097d640 1 Processor -- start 2024-04-25T01:08:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f832814aff0 0x7f83280cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f832806ebf0 0x7f832806c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832806cd90 0x7f8328070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f83280d0380 con 0x7f832814aff0 2024-04-25T01:08:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f832806d190 con 0x7f832806ebf0 2024-04-25T01:08:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f8330081640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f832814b3d0 con 0x7f832806cd90 2024-04-25T01:08:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f3395232640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3390162180 0x7f339006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f3395232640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33900b4320 con 0x7f3390161da0 2024-04-25T01:08:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f3395232640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33900b3a80 con 0x7f33900b2930 2024-04-25T01:08:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f3395232640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f339006f410 con 0x7f3390162180 2024-04-25T01:08:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb5097d640 1 -- start start 2024-04-25T01:08:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 Processor -- start 2024-04-25T01:08:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f832814aff0 0x7f83280cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f832814aff0 0x7f83280cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49900/0 (socket says 172.21.15.33:49900) 2024-04-25T01:08:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 -- 172.21.15.33:0/4124987928 learned_addr learned my addr 172.21.15.33:0/4124987928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832e61e640 1 --2- 172.21.15.33:0/4124987928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832806cd90 0x7f8328070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832d61c640 1 --2- 172.21.15.33:0/4124987928 >> v2:172.21.15.138:3300/0 conn(0x7f832806ebf0 0x7f832806c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7fb00a1b9640 1 Processor -- start 2024-04-25T01:08:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 -- 172.21.15.33:0/4124987928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832806cd90 msgr2=0x7f8328070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f338f59e640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f33900b2930 0x7f3390071090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:48522/0 (socket says 172.21.15.33:48522) 2024-04-25T01:08:43.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f338f59e640 1 -- 172.21.15.33:0/216400754 learned_addr learned my addr 172.21.15.33:0/216400754 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:43.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 -- start start 2024-04-25T01:08:43.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 --2- 172.21.15.33:0/4124987928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832806cd90 0x7f8328070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 -- 172.21.15.33:0/4124987928 >> v2:172.21.15.138:3300/0 conn(0x7f832806ebf0 msgr2=0x7f832806c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f338e59c640 1 --2- 172.21.15.33:0/216400754 >> v2:172.21.15.138:3300/0 conn(0x7f3390162180 0x7f339006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 --2- 172.21.15.33:0/4124987928 >> v2:172.21.15.138:3300/0 conn(0x7f832806ebf0 0x7f832806c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832de1d640 1 -- 172.21.15.33:0/4124987928 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8328165d50 con 0x7f832814aff0 2024-04-25T01:08:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f832d61c640 1 --2- 172.21.15.33:0/4124987928 >> v2:172.21.15.138:3300/0 conn(0x7f832806ebf0 0x7f832806c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb5097d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c06ebf0 0x7ffb4c06c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb5097d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffb4c06cd90 0x7ffb4c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb5097d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb4c0d0380 con 0x7ffb4c14aff0 2024-04-25T01:08:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb5097d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffb4c06d1e0 con 0x7ffb4c06cd90 2024-04-25T01:08:43.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb5097d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c0d0120 con 0x7ffb4c06ebf0 2024-04-25T01:08:43.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7fb00a1b9640 1 -- start start 2024-04-25T01:08:43.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb49d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c06ebf0 0x7ffb4c06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb49d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c06ebf0 0x7ffb4c06c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60590/0 (socket says 172.21.15.33:60590) 2024-04-25T01:08:43.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb4ad9d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffb4c06cd90 0x7ffb4c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7ffb49d9b640 1 -- 172.21.15.33:0/1806367881 learned_addr learned my addr 172.21.15.33:0/1806367881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f338e59c640 1 -- 172.21.15.33:0/216400754 >> v1:172.21.15.138:6789/0 conn(0x7f33900b2930 legacy=0x7f3390071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b140b1c60 0x7f4b140aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 0x7f4b1406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b141544d0 0x7f4b1406cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1415d980 con 0x7f4b140b1c60 2024-04-25T01:08:43.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1415dc60 con 0x7f4b141544d0 2024-04-25T01:08:43.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b13fff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1415df40 con 0x7f4b14154070 2024-04-25T01:08:43.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f338e59c640 1 -- 172.21.15.33:0/216400754 >> v1:172.21.15.33:6789/0 conn(0x7f3390161da0 legacy=0x7f339006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f338e59c640 1 -- 172.21.15.33:0/216400754 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33900b3d60 con 0x7f3390162180 2024-04-25T01:08:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7fb00a1b9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 0x7fb00415d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7fb00a1b9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0040d0a60 0x7fb004066f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7fb00a1b9640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb00415de50 con 0x7fb00406d430 2024-04-25T01:08:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7fb00a1b9640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb00415dc60 con 0x7fb0040d0a60 2024-04-25T01:08:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7fb00a1b9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0040ad3a0 con 0x7fb0040acfc0 2024-04-25T01:08:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.829+0000 7f4b12ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b140b1c60 0x7f4b140aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4b140b1c60 0x7f4b140aec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49906/0 (socket says 172.21.15.33:49906) 2024-04-25T01:08:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 -- 172.21.15.33:0/3635362740 learned_addr learned my addr 172.21.15.33:0/3635362740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7ffb49d9b640 1 -- 172.21.15.33:0/1806367881 >> v2:172.21.15.138:3300/0 conn(0x7ffb4c06cd90 msgr2=0x7ffb4c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7ffb49d9b640 1 --2- 172.21.15.33:0/1806367881 >> v2:172.21.15.138:3300/0 conn(0x7ffb4c06cd90 0x7ffb4c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7ffb49d9b640 1 -- 172.21.15.33:0/1806367881 >> v1:172.21.15.138:6789/0 conn(0x7ffb4c14aff0 legacy=0x7ffb4c0cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7ffb49d9b640 1 -- 172.21.15.33:0/1806367881 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c14b3d0 con 0x7ffb4c06ebf0 2024-04-25T01:08:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b137fe640 1 --2- 172.21.15.33:0/3635362740 >> v2:172.21.15.138:3300/0 conn(0x7f4b141544d0 0x7f4b1406cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b127fc640 1 --2- 172.21.15.33:0/3635362740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 0x7f4b1406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 -- 172.21.15.33:0/3635362740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 msgr2=0x7f4b1406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 --2- 172.21.15.33:0/3635362740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 0x7f4b1406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 -- 172.21.15.33:0/3635362740 >> v2:172.21.15.138:3300/0 conn(0x7f4b141544d0 msgr2=0x7f4b1406cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 --2- 172.21.15.33:0/3635362740 >> v2:172.21.15.138:3300/0 conn(0x7f4b141544d0 0x7f4b1406cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 -- 172.21.15.33:0/3635362740 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1415e220 con 0x7f4b140b1c60 2024-04-25T01:08:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0040d0a60 0x7fb004066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb0040d0a60 0x7fb004066f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49910/0 (socket says 172.21.15.33:49910) 2024-04-25T01:08:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 -- 172.21.15.33:0/3804879222 learned_addr learned my addr 172.21.15.33:0/3804879222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 -- 172.21.15.33:0/3804879222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 msgr2=0x7fb00415d720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:08:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 --2- 172.21.15.33:0/3804879222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 0x7fb00415d720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 -- 172.21.15.33:0/3804879222 >> v1:172.21.15.33:6789/0 conn(0x7fb00406d430 legacy=0x7fb00406b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 -- 172.21.15.33:0/3804879222 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00415e130 con 0x7fb0040d0a60 2024-04-25T01:08:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a10918640 1 Processor -- start 2024-04-25T01:08:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a10918640 1 -- start start 2024-04-25T01:08:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb99ffff640 1 Processor -- start 2024-04-25T01:08:43.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb99ffff640 1 -- start start 2024-04-25T01:08:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a10918640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a0c0b2d90 0x7f4a0c159b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a10918640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4a0c0aef00 0x7f4a0c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a10918640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0c0aeab0 con 0x7f4a0c0b1c60 2024-04-25T01:08:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a10918640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0c0b41b0 con 0x7f4a0c0b2d90 2024-04-25T01:08:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a10918640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0c0b4490 con 0x7f4a0c0aef00 2024-04-25T01:08:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f832de1d640 1 --2- 172.21.15.33:0/4124987928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832814aff0 0x7f83280cfa10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83280cf780 tx=0x7f831801c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5dd761ec5921c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f338e59c640 1 --2- 172.21.15.33:0/216400754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390162180 0x7f339006ca60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3380009740 tx=0x7f338001c2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9840872787d2afa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4b12ffd640 1 --2- 172.21.15.33:0/3635362740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 0x7f4b140aec70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b140b4040 tx=0x7f4b0801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95223e7d21e26ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7ffb49d9b640 1 --2- 172.21.15.33:0/1806367881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c06ebf0 0x7ffb4c06c850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009870 tx=0x7ffb4001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39dc60017ed54395 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a09d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4a0c0b2d90 0x7f4a0c159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a0a59c640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4a0c0b1c60 0x7f4a0c15e140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:48544/0 (socket says 172.21.15.33:48544) 2024-04-25T01:08:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a0a59c640 1 -- 172.21.15.33:0/3034577137 learned_addr learned my addr 172.21.15.33:0/3034577137 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f832ce1b640 1 -- 172.21.15.33:0/4124987928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180b0070 con 0x7f832814aff0 2024-04-25T01:08:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f832ce1b640 1 -- 172.21.15.33:0/4124987928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83180ab600 con 0x7f832814aff0 2024-04-25T01:08:43.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f832ce1b640 1 -- 172.21.15.33:0/4124987928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180ab9d0 con 0x7f832814aff0 2024-04-25T01:08:43.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f338dd9b640 1 -- 172.21.15.33:0/216400754 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33800b0070 con 0x7f3390162180 2024-04-25T01:08:43.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f338dd9b640 1 -- 172.21.15.33:0/216400754 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33800ab600 con 0x7f3390162180 2024-04-25T01:08:43.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f338dd9b640 1 -- 172.21.15.33:0/216400754 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33800ab9d0 con 0x7f3390162180 2024-04-25T01:08:43.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 -- 172.21.15.33:0/216400754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390162180 msgr2=0x7f339006ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 --2- 172.21.15.33:0/216400754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390162180 0x7f339006ca60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3380009740 tx=0x7f338001c2a0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb4959a640 1 -- 172.21.15.33:0/1806367881 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb400b1070 con 0x7ffb4c06ebf0 2024-04-25T01:08:43.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb4959a640 1 -- 172.21.15.33:0/1806367881 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb400ac620 con 0x7ffb4c06ebf0 2024-04-25T01:08:43.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb4959a640 1 -- 172.21.15.33:0/1806367881 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb400ac9f0 con 0x7ffb4c06ebf0 2024-04-25T01:08:43.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 -- 172.21.15.33:0/1806367881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c06ebf0 msgr2=0x7ffb4c06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 --2- 172.21.15.33:0/1806367881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c06ebf0 0x7ffb4c06c850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009870 tx=0x7ffb4001c4d0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fee15d6b640 1 Processor -- start 2024-04-25T01:08:43.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fee15d6b640 1 -- start start 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b11ffb640 1 -- 172.21.15.33:0/3635362740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080b1070 con 0x7f4b140b1c60 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b11ffb640 1 -- 172.21.15.33:0/3635362740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b080ac620 con 0x7f4b140b1c60 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b11ffb640 1 -- 172.21.15.33:0/3635362740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080ac9f0 con 0x7f4b140b1c60 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 -- 172.21.15.33:0/3635362740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 msgr2=0x7f4b140aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 --2- 172.21.15.33:0/3635362740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 0x7f4b140aec70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4b140b4040 tx=0x7f4b0801c4f0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 -- 172.21.15.33:0/3635362740 shutdown_connections 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 --2- 172.21.15.33:0/3635362740 >> v2:172.21.15.138:3300/0 conn(0x7f4b141544d0 0x7f4b1406cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 --2- 172.21.15.33:0/3635362740 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 0x7f4b1406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 --2- 172.21.15.33:0/3635362740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 0x7f4b140aec70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 -- 172.21.15.33:0/3635362740 >> 172.21.15.33:0/3635362740 conn(0x7f4b1406ba40 msgr2=0x7f4b140b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 -- 172.21.15.33:0/3635362740 shutdown_connections 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7f4a0ad9d640 1 --2- 172.21.15.33:0/3034577137 >> v2:172.21.15.138:3300/0 conn(0x7f4a0c0aef00 0x7f4a0c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4a0959a640 1 -- 172.21.15.33:0/3034577137 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3714750150 0 0) 0x7f4a0c0aeab0 con 0x7f4a0c0b1c60 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4a0959a640 1 -- 172.21.15.33:0/3034577137 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f4007940 con 0x7f4a0c0b1c60 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4a09d9b640 1 -- 172.21.15.33:0/3034577137 >> v2:172.21.15.138:3300/0 conn(0x7f4a0c0aef00 msgr2=0x7f4a0c06d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4a09d9b640 1 --2- 172.21.15.33:0/3034577137 >> v2:172.21.15.138:3300/0 conn(0x7f4a0c0aef00 0x7f4a0c06d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4a09d9b640 1 -- 172.21.15.33:0/3034577137 >> v1:172.21.15.138:6789/0 conn(0x7f4a0c0b1c60 legacy=0x7f4a0c15e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4a09d9b640 1 -- 172.21.15.33:0/3034577137 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c0b4770 con 0x7f4a0c0b2d90 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 -- 172.21.15.33:0/4124987928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832814aff0 msgr2=0x7f83280cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 --2- 172.21.15.33:0/4124987928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832814aff0 0x7f83280cfa10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f83280cf780 tx=0x7f831801c1a0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 -- 172.21.15.33:0/4124987928 shutdown_connections 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 --2- 172.21.15.33:0/4124987928 >> v2:172.21.15.138:3300/0 conn(0x7f832806ebf0 0x7f832806c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 --2- 172.21.15.33:0/4124987928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832806cd90 0x7f8328070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 --2- 172.21.15.33:0/4124987928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832814aff0 0x7f83280cfa10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 -- 172.21.15.33:0/4124987928 >> 172.21.15.33:0/4124987928 conn(0x7f832806b3e0 msgr2=0x7f832806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 -- 172.21.15.33:0/4124987928 shutdown_connections 2024-04-25T01:08:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 -- 172.21.15.33:0/4124987928 wait complete. 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.833+0000 7fb002ffd640 1 --2- 172.21.15.33:0/3804879222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0040d0a60 0x7fb004066f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f23c0df8c24539 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb0027fc640 1 -- 172.21.15.33:0/3804879222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80b0070 con 0x7fb0040d0a60 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb0027fc640 1 -- 172.21.15.33:0/3804879222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff80ab600 con 0x7fb0040d0a60 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb0027fc640 1 -- 172.21.15.33:0/3804879222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80ab9d0 con 0x7fb0040d0a60 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99ffff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9a015bf10 0x7fb9a00ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a015c2f0 0x7fb9a00cfec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99ffff640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a00cc410 con 0x7fb9a00d0400 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99ffff640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a00afd00 con 0x7fb9a015bf10 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99ffff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00aee40 con 0x7fb9a015c2f0 2024-04-25T01:08:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99effd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9a015bf10 0x7fb9a00ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99effd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb9a015bf10 0x7fb9a00ae900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52118/0 (socket says 172.21.15.33:52118) 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99effd640 1 -- 172.21.15.33:0/1751211322 learned_addr learned my addr 172.21.15.33:0/1751211322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a015c2f0 0x7fb9a00cfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 -- 172.21.15.33:0/1806367881 shutdown_connections 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 --2- 172.21.15.33:0/1806367881 >> v2:172.21.15.138:3300/0 conn(0x7ffb4c06cd90 0x7ffb4c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 --2- 172.21.15.33:0/1806367881 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c06ebf0 0x7ffb4c06c850 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 -- 172.21.15.33:0/1806367881 >> 172.21.15.33:0/1806367881 conn(0x7ffb4c06b3e0 msgr2=0x7ffb4c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 -- 172.21.15.33:0/216400754 shutdown_connections 2024-04-25T01:08:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 --2- 172.21.15.33:0/216400754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390162180 0x7f339006ca60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 -- 172.21.15.33:0/216400754 >> 172.21.15.33:0/216400754 conn(0x7f339006ba60 msgr2=0x7f33900b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 -- 172.21.15.33:0/1806367881 shutdown_connections 2024-04-25T01:08:43.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 -- 172.21.15.33:0/3635362740 wait complete. 2024-04-25T01:08:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 -- 172.21.15.33:0/216400754 shutdown_connections 2024-04-25T01:08:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 -- 172.21.15.33:0/3804879222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0040d0a60 msgr2=0x7fb004066f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 --2- 172.21.15.33:0/3804879222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0040d0a60 0x7fb004066f50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff801c4b0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 -- 172.21.15.33:0/3804879222 shutdown_connections 2024-04-25T01:08:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 --2- 172.21.15.33:0/3804879222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0040d0a60 0x7fb004066f50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 --2- 172.21.15.33:0/3804879222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 0x7fb00415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 -- 172.21.15.33:0/3804879222 >> 172.21.15.33:0/3804879222 conn(0x7fb0040b4eb0 msgr2=0x7fb0040b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99e7fc640 1 -- 172.21.15.33:0/1751211322 >> v2:172.21.15.138:3300/0 conn(0x7fb9a015bf10 msgr2=0x7fb9a00ae900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99e7fc640 1 --2- 172.21.15.33:0/1751211322 >> v2:172.21.15.138:3300/0 conn(0x7fb9a015bf10 0x7fb9a00ae900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99e7fc640 1 -- 172.21.15.33:0/1751211322 >> v1:172.21.15.33:6789/0 conn(0x7fb9a00d0400 legacy=0x7fb9a00b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb99e7fc640 1 -- 172.21.15.33:0/1751211322 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00af0f0 con 0x7fb9a015c2f0 2024-04-25T01:08:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 -- 172.21.15.33:0/1806367881 wait complete. 2024-04-25T01:08:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 Processor -- start 2024-04-25T01:08:43.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 -- 172.21.15.33:0/216400754 wait complete. 2024-04-25T01:08:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 Processor -- start 2024-04-25T01:08:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 -- 172.21.15.33:0/3804879222 shutdown_connections 2024-04-25T01:08:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4b13fff640 1 Processor -- start 2024-04-25T01:08:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 Processor -- start 2024-04-25T01:08:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f3395232640 1 -- start start 2024-04-25T01:08:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7ffb5097d640 1 -- start start 2024-04-25T01:08:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fee15d6b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee080df390 0x7fee08007190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fee15d6b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fee0804f5c0 0x7fee08038020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7fb00a1b9640 1 -- 172.21.15.33:0/3804879222 wait complete. 2024-04-25T01:08:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee15d6b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee08054390 con 0x7fee0804fa80 2024-04-25T01:08:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee15d6b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fee08005ea0 con 0x7fee0804f5c0 2024-04-25T01:08:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee15d6b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee08006d00 con 0x7fee080df390 2024-04-25T01:08:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee080df390 0x7fee08007190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee080df390 0x7fee08007190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60622/0 (socket says 172.21.15.33:60622) 2024-04-25T01:08:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0effd640 1 -- 172.21.15.33:0/3454274630 learned_addr learned my addr 172.21.15.33:0/3454274630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0ffff640 1 --2- 172.21.15.33:0/3454274630 >> v2:172.21.15.33:3300/0 conn(0x7fee0804f5c0 0x7fee08038020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f3395232640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900b2930 0x7f3390164120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f3395232640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390161da0 0x7f3390156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f3395232640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390162180 0x7f3390156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f3395232640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33900723c0 con 0x7f33900b2930 2024-04-25T01:08:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f3395232640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3390072240 con 0x7f3390161da0 2024-04-25T01:08:43.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f3395232640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f339006d3b0 con 0x7f3390162180 2024-04-25T01:08:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 -- start start 2024-04-25T01:08:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f4a09d9b640 1 --2- 172.21.15.33:0/3034577137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b2d90 0x7f4a0c159b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c06eec0 tx=0x7f49fc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4341f2c230842ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a0959a640 1 -- 172.21.15.33:0/3034577137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0b1070 con 0x7f4a0c0b2d90 2024-04-25T01:08:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a0959a640 1 -- 172.21.15.33:0/3034577137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc0ac410 con 0x7f4a0c0b2d90 2024-04-25T01:08:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a0959a640 1 -- 172.21.15.33:0/3034577137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc0ac820 con 0x7f4a0c0b2d90 2024-04-25T01:08:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.837+0000 7f8330081640 1 -- start start 2024-04-25T01:08:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f8330081640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832806cd90 0x7f8328147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f8330081640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832806ebf0 0x7f83280b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f8330081640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832814aff0 0x7f83280b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb5097d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb4c06cd90 0x7ffb4c15f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f8330081640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83280cff50 con 0x7f832806cd90 2024-04-25T01:08:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f8330081640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8328166ad0 con 0x7f832806ebf0 2024-04-25T01:08:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338f59e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390162180 0x7f3390156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900b2930 0x7f3390164120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338f59e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390162180 0x7f3390156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60632/0 (socket says 172.21.15.33:60632) 2024-04-25T01:08:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338f59e640 1 -- 172.21.15.33:0/3884530376 learned_addr learned my addr 172.21.15.33:0/3884530376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 Processor -- start 2024-04-25T01:08:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338e59c640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390161da0 0x7f3390156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 -- start start 2024-04-25T01:08:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99e7fc640 1 --2- 172.21.15.33:0/1751211322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a015c2f0 0x7fb9a00cfec0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb9900097c0 tx=0x7fb99001c280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c65041a664ff02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338ed9d640 1 -- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390162180 msgr2=0x7f3390156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338ed9d640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3390162180 0x7f3390156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338ed9d640 1 -- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390161da0 msgr2=0x7f3390156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338ed9d640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3390161da0 0x7f3390156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb5097d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb4c06ebf0 0x7ffb4c162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb5097d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c14aff0 0x7ffb4c0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb5097d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c0b35a0 con 0x7ffb4c06ebf0 2024-04-25T01:08:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb5097d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb4c166ad0 con 0x7ffb4c06cd90 2024-04-25T01:08:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb5097d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4c0b3720 con 0x7ffb4c14aff0 2024-04-25T01:08:43.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338ed9d640 1 -- 172.21.15.33:0/3884530376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3390165320 con 0x7f33900b2930 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f8330081640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83280d00d0 con 0x7f832814aff0 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832de1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832806cd90 0x7f8328147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832de1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832806cd90 0x7f8328147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49958/0 (socket says 172.21.15.33:49958) 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832de1d640 1 -- 172.21.15.33:0/1880877597 learned_addr learned my addr 172.21.15.33:0/1880877597 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832e61e640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832814aff0 0x7f83280b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832d61c640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832806ebf0 0x7f83280b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 -- 172.21.15.33:0/3034577137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b2d90 msgr2=0x7f4a0c159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 --2- 172.21.15.33:0/3034577137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b2d90 0x7f4a0c159b40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c06eec0 tx=0x7f49fc01c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832d61c640 1 -- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832814aff0 msgr2=0x7f83280b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832d61c640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f832814aff0 0x7f83280b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832d61c640 1 -- 172.21.15.33:0/1880877597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832806cd90 msgr2=0x7f8328147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832d61c640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832806cd90 0x7f8328147c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832d61c640 1 -- 172.21.15.33:0/1880877597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83280b18f0 con 0x7f832806ebf0 2024-04-25T01:08:43.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99dffb640 1 -- 172.21.15.33:0/1751211322 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900b0070 con 0x7fb9a015c2f0 2024-04-25T01:08:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 0x7f4b1415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 0x7f4b14146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b141544d0 0x7f4b140afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b140b43e0 con 0x7f4b140b1c60 2024-04-25T01:08:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b14166b80 con 0x7f4b141544d0 2024-04-25T01:08:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4b140b4560 con 0x7f4b14154070 2024-04-25T01:08:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832de1d640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832806cd90 0x7f8328147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 -- 172.21.15.33:0/3034577137 shutdown_connections 2024-04-25T01:08:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 --2- 172.21.15.33:0/3034577137 >> v2:172.21.15.138:3300/0 conn(0x7f4a0c0aef00 0x7f4a0c06d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 --2- 172.21.15.33:0/3034577137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b2d90 0x7f4a0c159b40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 -- 172.21.15.33:0/3034577137 >> 172.21.15.33:0/3034577137 conn(0x7f4a0c06ba60 msgr2=0x7f4a0c0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb4c06cd90 0x7ffb4c15f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4a59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb4c06cd90 0x7ffb4c15f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52136/0 (socket says 172.21.15.33:52136) 2024-04-25T01:08:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4ad9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c14aff0 0x7ffb4c0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4a59c640 1 -- 172.21.15.33:0/3250947974 learned_addr learned my addr 172.21.15.33:0/3250947974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99dffb640 1 -- 172.21.15.33:0/1751211322 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9900ab600 con 0x7fb9a015c2f0 2024-04-25T01:08:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99dffb640 1 -- 172.21.15.33:0/1751211322 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900ab9d0 con 0x7fb9a015c2f0 2024-04-25T01:08:43.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 -- 172.21.15.33:0/3034577137 shutdown_connections 2024-04-25T01:08:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00406d430 0x7fb0041484d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 0x7fb00415efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0040d0a60 0x7fb0040ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb00406dcd0 con 0x7fb0040d0a60 2024-04-25T01:08:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0040d0ec0 con 0x7fb00406d430 2024-04-25T01:08:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb00a1b9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb004067490 con 0x7fb0040acfc0 2024-04-25T01:08:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f338ed9d640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900b2930 0x7f3390164120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f338400c390 tx=0x7f338400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b137fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b141544d0 0x7f4b140afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b137fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b141544d0 0x7f4b140afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52150/0 (socket says 172.21.15.33:52150) 2024-04-25T01:08:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b137fe640 1 -- 172.21.15.33:0/1927292683 learned_addr learned my addr 172.21.15.33:0/1927292683 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b127fc640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 0x7f4b14146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b12ffd640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 0x7f4b1415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f832d61c640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832806ebf0 0x7f83280b3830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f832400e8e0 tx=0x7f832400eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00406d430 0x7fb0041484d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00406d430 0x7fb0041484d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52164/0 (socket says 172.21.15.33:52164) 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 -- 172.21.15.33:0/302449064 learned_addr learned my addr 172.21.15.33:0/302449064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4ad9d640 1 -- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb4c06cd90 msgr2=0x7ffb4c15f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4ad9d640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb4c06cd90 0x7ffb4c15f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4ad9d640 1 -- 172.21.15.33:0/3250947974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb4c06ebf0 msgr2=0x7ffb4c162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4ad9d640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb4c06ebf0 0x7ffb4c162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4ad9d640 1 -- 172.21.15.33:0/3250947974 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4c0af590 con 0x7ffb4c14aff0 2024-04-25T01:08:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb002ffd640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 0x7fb00415efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0ffff640 1 -- 172.21.15.33:0/3454274630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee080df390 msgr2=0x7fee08007190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0ffff640 1 --2- 172.21.15.33:0/3454274630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee080df390 0x7fee08007190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0ffff640 1 -- 172.21.15.33:0/3454274630 >> v1:172.21.15.138:6789/0 conn(0x7fee0804fa80 legacy=0x7fee0804fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fee0ffff640 1 -- 172.21.15.33:0/3454274630 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee080060f0 con 0x7fee0804f5c0 2024-04-25T01:08:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 -- 172.21.15.33:0/1751211322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a015c2f0 msgr2=0x7fb9a00cfec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 --2- 172.21.15.33:0/1751211322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a015c2f0 0x7fb9a00cfec0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb9900097c0 tx=0x7fb99001c280 comp rx=0 tx=0).stop 2024-04-25T01:08:43.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384017070 con 0x7f33900b2930 2024-04-25T01:08:43.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33840052d0 con 0x7f33900b2930 2024-04-25T01:08:43.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384003820 con 0x7f33900b2930 2024-04-25T01:08:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 -- 172.21.15.33:0/1751211322 shutdown_connections 2024-04-25T01:08:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 --2- 172.21.15.33:0/1751211322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a015c2f0 0x7fb9a00cfec0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 --2- 172.21.15.33:0/1751211322 >> v2:172.21.15.138:3300/0 conn(0x7fb9a015bf10 0x7fb9a00ae900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 -- 172.21.15.33:0/1751211322 >> 172.21.15.33:0/1751211322 conn(0x7fb9a00b3620 msgr2=0x7fb9a00b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 -- 172.21.15.33:0/1751211322 shutdown_connections 2024-04-25T01:08:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb99ffff640 1 -- 172.21.15.33:0/1751211322 wait complete. 2024-04-25T01:08:43.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7ffb4ad9d640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c14aff0 0x7ffb4c0b0630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffb44007cf0 tx=0x7ffb4400e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324017070 con 0x7f832806ebf0 2024-04-25T01:08:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f3395232640 1 -- 172.21.15.33:0/3884530376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3390150c20 con 0x7f33900b2930 2024-04-25T01:08:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b12ffd640 1 -- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 msgr2=0x7f4b14146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b12ffd640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4b14154070 0x7f4b14146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b12ffd640 1 -- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b141544d0 msgr2=0x7f4b140afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 -- 172.21.15.33:0/3034577137 wait complete. 2024-04-25T01:08:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4a10918640 1 Processor -- start 2024-04-25T01:08:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b12ffd640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b141544d0 0x7f4b140afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b12ffd640 1 -- 172.21.15.33:0/1927292683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1415a890 con 0x7f4b140b1c60 2024-04-25T01:08:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b12ffd640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 0x7f4b1415a220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ac000 tx=0x7f4b08019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080b1070 con 0x7f4b140b1c60 2024-04-25T01:08:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b080b2040 con 0x7f4b140b1c60 2024-04-25T01:08:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b080b6620 con 0x7f4b140b1c60 2024-04-25T01:08:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7f4b13fff640 1 -- 172.21.15.33:0/1927292683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1415ab70 con 0x7f4b140b1c60 2024-04-25T01:08:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83280b1bd0 con 0x7f832806ebf0 2024-04-25T01:08:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fee0ffff640 1 --2- 172.21.15.33:0/3454274630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 0x7fee08038020 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097c0 tx=0x7fedfc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99154f9f27ea0b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4400edd0 con 0x7ffb4c14aff0 2024-04-25T01:08:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb440039a0 con 0x7ffb4c14aff0 2024-04-25T01:08:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb44003d90 con 0x7ffb4c14aff0 2024-04-25T01:08:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f3395232640 1 -- 172.21.15.33:0/3884530376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3390164660 con 0x7f33900b2930 2024-04-25T01:08:43.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83280b2190 con 0x7f832806ebf0 2024-04-25T01:08:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832400c8e0 con 0x7f832806ebf0 2024-04-25T01:08:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832400ccd0 con 0x7f832806ebf0 2024-04-25T01:08:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4b13fff640 1 -- 172.21.15.33:0/1927292683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1415b130 con 0x7f4b140b1c60 2024-04-25T01:08:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 -- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 msgr2=0x7fb00415efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb0040acfc0 0x7fb00415efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 -- 172.21.15.33:0/302449064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0040d0a60 msgr2=0x7fb0040ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0040d0a60 0x7fb0040ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.841+0000 7fb0037fe640 1 -- 172.21.15.33:0/302449064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00414b4e0 con 0x7fb00406d430 2024-04-25T01:08:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f3395232640 1 -- 172.21.15.33:0/3884530376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3390150a10 con 0x7f33900b2930 2024-04-25T01:08:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8310001040 con 0x7f832806ebf0 2024-04-25T01:08:43.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 Processor -- start 2024-04-25T01:08:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 -- start start 2024-04-25T01:08:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb0037fe640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00406d430 0x7fb0041484d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faff400b4d0 tx=0x7faff400b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fee0e7fc640 1 -- 172.21.15.33:0/3454274630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0ab5f0 con 0x7fee0804f5c0 2024-04-25T01:08:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a10918640 1 -- start start 2024-04-25T01:08:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a10918640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a0c0aef00 0x7f4a0c1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a10918640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b1c60 0x7f4a0c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a10918640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0c0b2d90 0x7f4a0c1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a10918640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c166e60 con 0x7f4a0c0b1c60 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a10918640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c06db20 con 0x7f4a0c0b2d90 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a10918640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0c166fe0 con 0x7f4a0c0aef00 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a0c0aef00 0x7f4a0c1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a0c0aef00 0x7f4a0c1504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60676/0 (socket says 172.21.15.33:60676) 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 -- 172.21.15.33:0/2280091893 learned_addr learned my addr 172.21.15.33:0/2280091893 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a09d9b640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b1c60 0x7f4a0c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0ad9d640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0c0b2d90 0x7f4a0c1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4014070 con 0x7fb00406d430 2024-04-25T01:08:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb004160f30 con 0x7fb00406d430 2024-04-25T01:08:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0041617d0 con 0x7fb00406d430 2024-04-25T01:08:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4aec001040 con 0x7f4b140b1c60 2024-04-25T01:08:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff4010890 con 0x7fb00406d430 2024-04-25T01:08:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff4010c80 con 0x7fb00406d430 2024-04-25T01:08:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fee0e7fc640 1 -- 172.21.15.33:0/3454274630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedfc0b0070 con 0x7fee0804f5c0 2024-04-25T01:08:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fee0e7fc640 1 -- 172.21.15.33:0/3454274630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedfc0ab9b0 con 0x7fee0804f5c0 2024-04-25T01:08:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a00d0400 0x7fb9a00c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fafe4001040 con 0x7fb00406d430 2024-04-25T01:08:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a015bf10 0x7fb9a0153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a015c2f0 0x7fb9a0158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7ffb5097d640 1 -- 172.21.15.33:0/3250947974 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4c0af870 con 0x7ffb4c14aff0 2024-04-25T01:08:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7ffb5097d640 1 -- 172.21.15.33:0/3250947974 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb4c0afe30 con 0x7ffb4c14aff0 2024-04-25T01:08:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a00d0400 0x7fb9a00c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99effd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a00d0400 0x7fb9a00c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60684/0 (socket says 172.21.15.33:60684) 2024-04-25T01:08:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a015bf10 0x7fb9a0153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99e7fc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a015bf10 0x7fb9a0153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52180/0 (socket says 172.21.15.33:52180) 2024-04-25T01:08:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fee15d6b640 1 -- 172.21.15.33:0/3454274630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 msgr2=0x7fee08038020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fee15d6b640 1 --2- 172.21.15.33:0/3454274630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 0x7fee08038020 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097c0 tx=0x7fedfc01c430 comp rx=0 tx=0).stop 2024-04-25T01:08:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a015c2f0 0x7fb9a0158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a015c2f0 0x7fb9a0158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49986/0 (socket says 172.21.15.33:49986) 2024-04-25T01:08:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f832401a070 con 0x7f832806ebf0 2024-04-25T01:08:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f830c067940 0x7f830c069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8324085f80 con 0x7f832806ebf0 2024-04-25T01:08:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f830c0710c0 0x7f830c073500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 --> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f832400e020 con 0x7f830c0710c0 2024-04-25T01:08:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8324086460 con 0x7f832806ebf0 2024-04-25T01:08:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f338401a070 con 0x7f33900b2930 2024-04-25T01:08:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99effd640 1 -- 172.21.15.33:0/1412546348 learned_addr learned my addr 172.21.15.33:0/1412546348 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00b1230 con 0x7fb9a015c2f0 2024-04-25T01:08:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00b10b0 con 0x7fb9a015bf10 2024-04-25T01:08:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7fb99ffff640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00b13b0 con 0x7fb9a00d0400 2024-04-25T01:08:43.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f832e61e640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f830c0710c0 0x7f830c073500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- 172.21.15.33:0/3454274630 shutdown_connections 2024-04-25T01:08:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 --2- 172.21.15.33:0/3454274630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 0x7fee08038020 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 --2- 172.21.15.33:0/3454274630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee080df390 0x7fee08007190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- 172.21.15.33:0/3454274630 >> 172.21.15.33:0/3454274630 conn(0x7fee08009fc0 msgr2=0x7fee0800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- 172.21.15.33:0/3454274630 shutdown_connections 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b080b67c0 con 0x7f4b140b1c60 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4afbfff640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4b00067990 0x7f4b00069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4b081207e0 con 0x7f4b140b1c60 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4afbfff640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f4b00071080 0x7f4b000734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 --> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b080c0e70 con 0x7f4b00071080 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4b127fc640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4b00067990 0x7f4b00069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f33777fe640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f337c067910 0x7f337c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3384085820 con 0x7f33900b2930 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f33777fe640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f337c071090 0x7f337c0734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 --> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3384009e50 con 0x7f337c071090 2024-04-25T01:08:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3384085d00 con 0x7f33900b2930 2024-04-25T01:08:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f338e59c640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f337c067910 0x7f337c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f338f59e640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f337c071090 0x7f337c0734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7ffb5097d640 1 -- 172.21.15.33:0/3250947974 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb4c0b0210 con 0x7ffb4c14aff0 2024-04-25T01:08:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4b137fe640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f4b00071080 0x7f4b000734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f832de1d640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f830c067940 0x7f830c069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f338f59e640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f337c071090 0x7f337c0734d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- 172.21.15.33:0/3454274630 wait complete. 2024-04-25T01:08:43.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4b137fe640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f4b00071080 0x7f4b000734c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 -- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0c0b2d90 msgr2=0x7f4a0c1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a0c0b2d90 0x7f4a0c1527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 -- 172.21.15.33:0/2280091893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b1c60 msgr2=0x7f4a0c14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4a0c0b1c60 0x7f4a0c14aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.845+0000 7f4a0a59c640 1 -- 172.21.15.33:0/2280091893 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c14bf10 con 0x7f4a0c0aef00 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4a0a59c640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a0c0aef00 0x7f4a0c1504d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400cbc0 tx=0x7f4a0400a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04017070 con 0x7f4a0c0aef00 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0400ad40 con 0x7f4a0c0aef00 2024-04-25T01:08:43.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04012620 con 0x7f4a0c0aef00 2024-04-25T01:08:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb99f7fe640 1 -- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a00d0400 msgr2=0x7fb9a00c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb99f7fe640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9a00d0400 0x7fb9a00c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb99f7fe640 1 -- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a015bf10 msgr2=0x7fb9a0153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb99f7fe640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9a015bf10 0x7fb9a0153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb99f7fe640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00c5ae0 con 0x7fb9a015c2f0 2024-04-25T01:08:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4b127fc640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4b00067990 0x7f4b00069e50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b140b0840 tx=0x7f4b04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 Processor -- start 2024-04-25T01:08:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f832de1d640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f830c067940 0x7f830c069e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8318000c10 tx=0x7f83180a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f338e59c640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f337c067910 0x7f337c069dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f33800ad000 tx=0x7f33800a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c14c1f0 con 0x7f4a0c0aef00 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c14c7b0 con 0x7f4a0c0aef00 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a0c04f580 con 0x7f4a0c0aef00 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faff4017070 con 0x7fb00406d430 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fafebfff640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff00678c0 0x7faff0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faff40862c0 con 0x7fb00406d430 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fafebfff640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7faff0071040 0x7faff0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 --> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faff0073b50 con 0x7faff0071040 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faff40866b0 con 0x7fb00406d430 2024-04-25T01:08:43.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb002ffd640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff00678c0 0x7faff0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4b080ecca0 con 0x7f4b140b1c60 2024-04-25T01:08:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== osd.3 v2:172.21.15.33:6816/2195826364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b080c0e70 con 0x7f4b00071080 2024-04-25T01:08:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb99f7fe640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a015c2f0 0x7fb9a0158020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb99800be20 tx=0x7fb99800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- start start 2024-04-25T01:08:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 0x7fee08049300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== osd.0 v2:172.21.15.33:6824/1142279268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3384009e50 con 0x7f337c071090 2024-04-25T01:08:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee0804fa80 0x7fee080da990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee080df390 0x7fee0804c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee080076d0 con 0x7fee0804f5c0 2024-04-25T01:08:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee080ebfd0 con 0x7fee080df390 2024-04-25T01:08:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee15d6b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee080d1d10 con 0x7fee0804fa80 2024-04-25T01:08:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7f832e61e640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f830c0710c0 0x7f830c073500 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 0x7fee08049300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 0x7fee08049300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49990/0 (socket says 172.21.15.33:49990) 2024-04-25T01:08:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee0f7fe640 1 -- 172.21.15.33:0/992227511 learned_addr learned my addr 172.21.15.33:0/992227511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:43.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb003fff640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7faff0071040 0x7faff0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb4401b020 con 0x7ffb4c14aff0 2024-04-25T01:08:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7ffb3affd640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb2c067910 0x7ffb2c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffb44086960 con 0x7ffb4c14aff0 2024-04-25T01:08:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7ffb3affd640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7ffb2c071080 0x7ffb2c0734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 --> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb44004860 con 0x7ffb2c071080 2024-04-25T01:08:43.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ffb44086e40 con 0x7ffb4c14aff0 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee0effd640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee0804fa80 0x7fee080da990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fee0ffff640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee080df390 0x7fee0804c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee0effd640 1 -- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee080df390 msgr2=0x7fee0804c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee0effd640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee080df390 0x7fee0804c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee0effd640 1 -- 172.21.15.33:0/992227511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 msgr2=0x7fee08049300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee0effd640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee0804f5c0 0x7fee08049300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee0effd640 1 -- 172.21.15.33:0/992227511 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0804d010 con 0x7fee0804fa80 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee0effd640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee0804fa80 0x7fee080da990 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee1006a000 tx=0x7fee10068580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10073b40 con 0x7fee0804fa80 2024-04-25T01:08:43.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee10073ce0 con 0x7fee0804fa80 2024-04-25T01:08:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb9877fe640 1 -- 172.21.15.33:0/1412546348 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998019070 con 0x7fb9a015c2f0 2024-04-25T01:08:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb9877fe640 1 -- 172.21.15.33:0/1412546348 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb998014440 con 0x7fb9a015c2f0 2024-04-25T01:08:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb9877fe640 1 -- 172.21.15.33:0/1412546348 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998014870 con 0x7fb9a015c2f0 2024-04-25T01:08:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.849+0000 7fb99ffff640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a0158b20 con 0x7fb9a015c2f0 2024-04-25T01:08:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== osd.5 v2:172.21.15.138:6801/1955200876 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f832400e020 con 0x7f830c0710c0 2024-04-25T01:08:43.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7ffb4a59c640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb2c067910 0x7ffb2c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7ffb4a59c640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb2c067910 0x7ffb2c069dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0cea80 tx=0x7ffb3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fb9877fe640 1 -- 172.21.15.33:0/1412546348 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb99801c070 con 0x7fb9a015c2f0 2024-04-25T01:08:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fb9877fe640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb98c067910 0x7fb98c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee100740d0 con 0x7fee0804fa80 2024-04-25T01:08:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee15d6b640 1 -- 172.21.15.33:0/992227511 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0804d2f0 con 0x7fee0804fa80 2024-04-25T01:08:43.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee15d6b640 1 -- 172.21.15.33:0/992227511 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0804cd20 con 0x7fee0804fa80 2024-04-25T01:08:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fb99ffff640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a0154c90 con 0x7fb9a015c2f0 2024-04-25T01:08:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fb9877fe640 1 -- 172.21.15.33:0/1412546348 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb998053f10 con 0x7fb9a015c2f0 2024-04-25T01:08:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fb99effd640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb98c067910 0x7fb98c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fee15d6b640 1 -- 172.21.15.33:0/992227511 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee080053a0 con 0x7fee0804fa80 2024-04-25T01:08:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7fb99effd640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb98c067910 0x7fb98c069dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb994006f50 tx=0x7fb994009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4a0401a070 con 0x7f4a0c0aef00 2024-04-25T01:08:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f49faffd640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49ec067910 0x7f49ec069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4a040863f0 con 0x7f4a0c0aef00 2024-04-25T01:08:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f49faffd640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f49ec071090 0x7f49ec0734d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 --> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49ec073ba0 con 0x7f49ec071090 2024-04-25T01:08:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4a040868d0 con 0x7f4a0c0aef00 2024-04-25T01:08:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f4a0ad9d640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f49ec071090 0x7f49ec0734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.853+0000 7f4a0ad9d640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f49ec071090 0x7f49ec0734d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fb002ffd640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff00678c0 0x7faff0069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faff80ad000 tx=0x7faff8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7ffb49d9b640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7ffb2c071080 0x7ffb2c0734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7f4a09d9b640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49ec067910 0x7f49ec069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7ffb49d9b640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7ffb2c071080 0x7ffb2c0734c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fee10074660 con 0x7fee0804fa80 2024-04-25T01:08:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== osd.4 v2:172.21.15.138:6816/100119801 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49ec073ba0 con 0x7f49ec071090 2024-04-25T01:08:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fb9857fa640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7fb9a0154e60 0x7fb9a0155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb9a004f580 con 0x7fb9a0154e60 2024-04-25T01:08:43.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fb99e7fc640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7fb9a0154e60 0x7fb9a0155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7f4a09d9b640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49ec067910 0x7f49ec069dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ae000 tx=0x7f49fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fb003fff640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7faff0071040 0x7faff0073480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fedfbfff640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedf40678c0 0x7fedf4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fee10100880 con 0x7fee0804fa80 2024-04-25T01:08:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fedfbfff640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7fedf4070fc0 0x7fedf4073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 --> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee10090ec0 con 0x7fedf4070fc0 2024-04-25T01:08:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee10100d10 con 0x7fee0804fa80 2024-04-25T01:08:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fee0ffff640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7fedf4070fc0 0x7fedf4073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fb99e7fc640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7fb9a0154e60 0x7fb9a0155750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== osd.1 v2:172.21.15.33:6808/1105804411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb44004860 con 0x7ffb2c071080 2024-04-25T01:08:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fee0f7fe640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedf40678c0 0x7fedf4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fee0ffff640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7fedf4070fc0 0x7fedf4073400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== osd.2 v2:172.21.15.33:6800/1000748225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee10090ec0 con 0x7fedf4070fc0 2024-04-25T01:08:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fee0f7fe640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedf40678c0 0x7fedf4069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fee04001960 tx=0x7fee04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:43.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fb9877fe640 1 -- 172.21.15.33:0/1412546348 <== osd.7 v2:172.21.15.138:6800/400031455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb9a004f580 con 0x7fb9a0154e60 2024-04-25T01:08:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.857+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== osd.6 v2:172.21.15.138:6824/1315598862 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faff0073b50 con 0x7faff0071040 2024-04-25T01:08:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.877+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 --> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8310002de0 con 0x7f830c0710c0 2024-04-25T01:08:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.877+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 --> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb9a00cfa90 con 0x7fb9a0154e60 2024-04-25T01:08:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f831e7fc640 1 -- 172.21.15.33:0/1880877597 <== osd.5 v2:172.21.15.138:6801/1955200876 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8310002de0 con 0x7f830c0710c0 2024-04-25T01:08:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9877fe640 1 -- 172.21.15.33:0/1412546348 <== osd.7 v2:172.21.15.138:6800/400031455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb9a00cfa90 con 0x7fb9a0154e60 2024-04-25T01:08:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 --> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4aec002d60 con 0x7f4b00071080 2024-04-25T01:08:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7fb9a0154e60 msgr2=0x7fb9a0155750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7fb9a0154e60 0x7fb9a0155750 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f830c0710c0 msgr2=0x7f830c073500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f830c0710c0 0x7f830c073500 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb98c067910 msgr2=0x7fb98c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb98c067910 0x7fb98c069dd0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb994006f50 tx=0x7fb994009240 comp rx=0 tx=0).stop 2024-04-25T01:08:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a015c2f0 msgr2=0x7fb9a0158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 --2- 172.21.15.33:0/1412546348 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9a015c2f0 0x7fb9a0158020 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb99800be20 tx=0x7fb99800bf10 comp rx=0 tx=0).stop 2024-04-25T01:08:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4afbfff640 1 -- 172.21.15.33:0/1927292683 <== osd.3 v2:172.21.15.33:6816/2195826364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4aec002d60 con 0x7f4b00071080 2024-04-25T01:08:43.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f830c067940 msgr2=0x7f830c069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f830c067940 0x7f830c069e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8318000c10 tx=0x7f83180a6000 comp rx=0 tx=0).stop 2024-04-25T01:08:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832806ebf0 msgr2=0x7f83280b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 --2- 172.21.15.33:0/1880877597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832806ebf0 0x7f83280b3830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f832400e8e0 tx=0x7f832400eda0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb99f7fe640 1 -- 172.21.15.33:0/1412546348 reap_dead start 2024-04-25T01:08:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f832e61e640 1 -- 172.21.15.33:0/1880877597 reap_dead start 2024-04-25T01:08:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f3395232640 1 -- 172.21.15.33:0/3884530376 --> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3390151420 con 0x7f337c071090 2024-04-25T01:08:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 shutdown_connections 2024-04-25T01:08:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 >> 172.21.15.33:0/1880877597 conn(0x7f832806b3e0 msgr2=0x7f832806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 shutdown_connections 2024-04-25T01:08:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f8330081640 1 -- 172.21.15.33:0/1880877597 wait complete. 2024-04-25T01:08:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 shutdown_connections 2024-04-25T01:08:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 >> 172.21.15.33:0/1412546348 conn(0x7fb9a00b3620 msgr2=0x7fb9a015abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33777fe640 1 -- 172.21.15.33:0/3884530376 <== osd.0 v2:172.21.15.33:6824/1142279268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3390151420 con 0x7f337c071090 2024-04-25T01:08:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f4b00071080 msgr2=0x7f4b000734c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f4b00071080 0x7f4b000734c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 shutdown_connections 2024-04-25T01:08:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7fb9857fa640 1 -- 172.21.15.33:0/1412546348 wait complete. 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4b00067990 msgr2=0x7f4b00069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4b00067990 0x7f4b00069e50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4b140b0840 tx=0x7f4b04008040 comp rx=0 tx=0).stop 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 msgr2=0x7f4b1415a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 --2- 172.21.15.33:0/1927292683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b140b1c60 0x7f4b1415a220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b080ac000 tx=0x7f4b08019290 comp rx=0 tx=0).stop 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4b137fe640 1 -- 172.21.15.33:0/1927292683 reap_dead start 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 shutdown_connections 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 >> 172.21.15.33:0/1927292683 conn(0x7f4b1406ba40 msgr2=0x7f4b1406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 shutdown_connections 2024-04-25T01:08:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 -- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f337c071090 msgr2=0x7f337c0734d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f4af3fff640 1 -- 172.21.15.33:0/1927292683 wait complete. 2024-04-25T01:08:43.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f337c071090 0x7f337c0734d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 -- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f337c067910 msgr2=0x7f337c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f337c067910 0x7f337c069dd0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f33800ad000 tx=0x7f33800a6000 comp rx=0 tx=0).stop 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 -- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900b2930 msgr2=0x7f3390164120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 --2- 172.21.15.33:0/3884530376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33900b2930 0x7f3390164120 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f338400c390 tx=0x7f338400c850 comp rx=0 tx=0).stop 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f338f59e640 1 -- 172.21.15.33:0/3884530376 reap_dead start 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 -- 172.21.15.33:0/3884530376 shutdown_connections 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.881+0000 7f33757fa640 1 -- 172.21.15.33:0/3884530376 >> 172.21.15.33:0/3884530376 conn(0x7f339006ba60 msgr2=0x7f33900aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.885+0000 7f33757fa640 1 -- 172.21.15.33:0/3884530376 shutdown_connections 2024-04-25T01:08:43.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.885+0000 7f33757fa640 1 -- 172.21.15.33:0/3884530376 wait complete. 2024-04-25T01:08:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.885+0000 7ffb5097d640 1 -- 172.21.15.33:0/3250947974 --> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb4c04f580 con 0x7ffb2c071080 2024-04-25T01:08:43.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.885+0000 7ffb3affd640 1 -- 172.21.15.33:0/3250947974 <== osd.1 v2:172.21.15.33:6808/1105804411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ffb4c04f580 con 0x7ffb2c071080 2024-04-25T01:08:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 -- 172.21.15.33:0/3250947974 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7ffb2c071080 msgr2=0x7ffb2c0734c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7ffb2c071080 0x7ffb2c0734c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 -- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb2c067910 msgr2=0x7ffb2c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb2c067910 0x7ffb2c069dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c0cea80 tx=0x7ffb3c009040 comp rx=0 tx=0).stop 2024-04-25T01:08:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 --> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fafe4002d60 con 0x7faff0071040 2024-04-25T01:08:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 -- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c14aff0 msgr2=0x7ffb4c0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 --2- 172.21.15.33:0/3250947974 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb4c14aff0 0x7ffb4c0b0630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffb44007cf0 tx=0x7ffb4400e4b0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb4ad9d640 1 -- 172.21.15.33:0/3250947974 reap_dead start 2024-04-25T01:08:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fee15d6b640 1 -- 172.21.15.33:0/992227511 --> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fee0804f190 con 0x7fedf4070fc0 2024-04-25T01:08:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 -- 172.21.15.33:0/3250947974 shutdown_connections 2024-04-25T01:08:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 -- 172.21.15.33:0/3250947974 >> 172.21.15.33:0/3250947974 conn(0x7ffb4c06b3e0 msgr2=0x7ffb4c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 -- 172.21.15.33:0/3250947974 shutdown_connections 2024-04-25T01:08:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7ffb38ff9640 1 -- 172.21.15.33:0/3250947974 wait complete. 2024-04-25T01:08:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fedfbfff640 1 -- 172.21.15.33:0/992227511 <== osd.2 v2:172.21.15.33:6800/1000748225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fee0804f190 con 0x7fedf4070fc0 2024-04-25T01:08:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fafebfff640 1 -- 172.21.15.33:0/302449064 <== osd.6 v2:172.21.15.138:6824/1315598862 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fafe4002d60 con 0x7faff0071040 2024-04-25T01:08:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7faff0071040 msgr2=0x7faff0073480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7faff0071040 0x7faff0073480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fedf9ffb640 1 -- 172.21.15.33:0/992227511 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7fedf4070fc0 msgr2=0x7fedf4073400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fedf9ffb640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7fedf4070fc0 0x7fedf4073400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff00678c0 msgr2=0x7faff0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff00678c0 0x7faff0069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faff80ad000 tx=0x7faff8008040 comp rx=0 tx=0).stop 2024-04-25T01:08:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fedf9ffb640 1 -- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedf40678c0 msgr2=0x7fedf4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00406d430 msgr2=0x7fb0041484d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 --2- 172.21.15.33:0/302449064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb00406d430 0x7fb0041484d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faff400b4d0 tx=0x7faff400b990 comp rx=0 tx=0).stop 2024-04-25T01:08:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fedf9ffb640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedf40678c0 0x7fedf4069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fee04001960 tx=0x7fee04009040 comp rx=0 tx=0).stop 2024-04-25T01:08:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb003fff640 1 -- 172.21.15.33:0/302449064 reap_dead start 2024-04-25T01:08:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 shutdown_connections 2024-04-25T01:08:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fedf9ffb640 1 -- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee0804fa80 msgr2=0x7fee080da990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 >> 172.21.15.33:0/302449064 conn(0x7fb0040b4eb0 msgr2=0x7fb00415c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fedf9ffb640 1 --2- 172.21.15.33:0/992227511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee0804fa80 0x7fee080da990 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fee1006a000 tx=0x7fee10068580 comp rx=0 tx=0).stop 2024-04-25T01:08:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 shutdown_connections 2024-04-25T01:08:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.889+0000 7fb00a1b9640 1 -- 172.21.15.33:0/302449064 wait complete. 2024-04-25T01:08:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.893+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 --> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a0c0cf550 con 0x7f49ec071090 2024-04-25T01:08:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.893+0000 7f49faffd640 1 -- 172.21.15.33:0/2280091893 <== osd.4 v2:172.21.15.138:6816/100119801 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a0c0cf550 con 0x7f49ec071090 2024-04-25T01:08:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.893+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f49ec071090 msgr2=0x7f49ec0734d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.893+0000 7f4a10918640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f49ec071090 0x7f49ec0734d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49ec067910 msgr2=0x7f49ec069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f49ec067910 0x7f49ec069dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49fc0ae000 tx=0x7f49fc008040 comp rx=0 tx=0).stop 2024-04-25T01:08:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a0c0aef00 msgr2=0x7f4a0c1504d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 --2- 172.21.15.33:0/2280091893 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4a0c0aef00 0x7f4a0c1504d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400cbc0 tx=0x7f4a0400a430 comp rx=0 tx=0).stop 2024-04-25T01:08:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a0ad9d640 1 -- 172.21.15.33:0/2280091893 reap_dead start 2024-04-25T01:08:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 shutdown_connections 2024-04-25T01:08:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 >> 172.21.15.33:0/2280091893 conn(0x7f4a0c06ba60 msgr2=0x7f4a0c158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 shutdown_connections 2024-04-25T01:08:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.897+0000 7f4a10918640 1 -- 172.21.15.33:0/2280091893 wait complete. 2024-04-25T01:08:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.905+0000 7fee0ffff640 1 -- 172.21.15.33:0/992227511 reap_dead start 2024-04-25T01:08:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.905+0000 7fedf9ffb640 1 -- 172.21.15.33:0/992227511 shutdown_connections 2024-04-25T01:08:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.905+0000 7fedf9ffb640 1 -- 172.21.15.33:0/992227511 >> 172.21.15.33:0/992227511 conn(0x7fee08009fc0 msgr2=0x7fee08170ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.905+0000 7fedf9ffb640 1 -- 172.21.15.33:0/992227511 shutdown_connections 2024-04-25T01:08:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:43.905+0000 7fedf9ffb640 1 -- 172.21.15.33:0/992227511 wait complete. 2024-04-25T01:08:43.944 INFO:teuthology.orchestra.run.smithi033.stdout:64424509449 2024-04-25T01:08:43.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T01:08:43.945 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2024-04-25T01:08:43.946 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2024-04-25T01:08:43.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T01:08:43.947 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T01:08:43.949 INFO:teuthology.orchestra.run.smithi033.stdout:64424509448 2024-04-25T01:08:43.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T01:08:43.951 INFO:teuthology.orchestra.run.smithi033.stdout:64424509449 2024-04-25T01:08:43.951 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T01:08:43.952 INFO:teuthology.orchestra.run.smithi033.stdout:64424509449 2024-04-25T01:08:43.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T01:08:43.954 INFO:teuthology.orchestra.run.smithi033.stdout:64424509450 2024-04-25T01:08:43.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T01:08:43.958 INFO:teuthology.orchestra.run.smithi033.stdout:64424509447 2024-04-25T01:08:43.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T01:08:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.161+0000 7f48d0e5d640 1 Processor -- start 2024-04-25T01:08:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.161+0000 7f48d0e5d640 1 -- start start 2024-04-25T01:08:44.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48d0e5d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48cc06d8e0 0x7f48cc070db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48d0e5d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f48cc0b2d90 0x7f48cc0712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48d0e5d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48cc0b3f30 con 0x7f48cc159da0 2024-04-25T01:08:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48d0e5d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f48cc15a180 con 0x7f48cc06d8e0 2024-04-25T01:08:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48d0e5d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f48cc0aeab0 con 0x7f48cc0b2d90 2024-04-25T01:08:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48c9d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48cc06d8e0 0x7f48cc070db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48c9d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f48cc06d8e0 0x7f48cc070db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50000/0 (socket says 172.21.15.33:50000) 2024-04-25T01:08:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48c9d9b640 1 -- 172.21.15.33:0/3917457191 learned_addr learned my addr 172.21.15.33:0/3917457191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48cad9d640 1 --2- 172.21.15.33:0/3917457191 >> v2:172.21.15.138:3300/0 conn(0x7f48cc0b2d90 0x7f48cc0712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48c9d9b640 1 -- 172.21.15.33:0/3917457191 >> v2:172.21.15.138:3300/0 conn(0x7f48cc0b2d90 msgr2=0x7f48cc0712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48c9d9b640 1 --2- 172.21.15.33:0/3917457191 >> v2:172.21.15.138:3300/0 conn(0x7f48cc0b2d90 0x7f48cc0712f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48c9d9b640 1 -- 172.21.15.33:0/3917457191 >> v1:172.21.15.138:6789/0 conn(0x7f48cc159da0 legacy=0x7f48cc06d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.165+0000 7f48c9d9b640 1 -- 172.21.15.33:0/3917457191 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc0b41b0 con 0x7f48cc06d8e0 2024-04-25T01:08:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48c9d9b640 1 --2- 172.21.15.33:0/3917457191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc06d8e0 0x7f48cc070db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0ae5c0 tx=0x7f48c401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcf94f5433b0da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc5181a4640 1 Processor -- start 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc5181a4640 1 -- start start 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc5181a4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 0x7fc51006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc5181a4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc510071730 con 0x7fc51006cfa0 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc5181a4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc510071a10 con 0x7fc51008bb30 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc5181a4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc5100714e0 con 0x7fc5100b2850 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc516741640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc51006cfa0 0x7fc51016adf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33900/0 (socket says 172.21.15.33:33900) 2024-04-25T01:08:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc516741640 1 -- 172.21.15.33:0/3896286340 learned_addr learned my addr 172.21.15.33:0/3896286340 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48c959a640 1 -- 172.21.15.33:0/3917457191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40ac070 con 0x7f48cc06d8e0 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48c959a640 1 -- 172.21.15.33:0/3917457191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40a8450 con 0x7f48cc06d8e0 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48c959a640 1 -- 172.21.15.33:0/3917457191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40a8820 con 0x7f48cc06d8e0 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 -- 172.21.15.33:0/3917457191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc06d8e0 msgr2=0x7f48cc070db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 --2- 172.21.15.33:0/3917457191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc06d8e0 0x7f48cc070db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0ae5c0 tx=0x7f48c401c740 comp rx=0 tx=0).stop 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 -- 172.21.15.33:0/3917457191 shutdown_connections 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 --2- 172.21.15.33:0/3917457191 >> v2:172.21.15.138:3300/0 conn(0x7f48cc0b2d90 0x7f48cc0712f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 --2- 172.21.15.33:0/3917457191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc06d8e0 0x7f48cc070db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 -- 172.21.15.33:0/3917457191 >> 172.21.15.33:0/3917457191 conn(0x7f48cc06ba60 msgr2=0x7f48cc0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 -- 172.21.15.33:0/3917457191 shutdown_connections 2024-04-25T01:08:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7f48d0e5d640 1 -- 172.21.15.33:0/3917457191 wait complete. 2024-04-25T01:08:44.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc51573f640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 0x7fc51006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3238639836 0 0) 0x7fc510071730 con 0x7fc51006cfa0 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4fc005740 con 0x7fc51006cfa0 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2745505274 0 0) 0x7fc4fc005740 con 0x7fc51006cfa0 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 msgr2=0x7fc51006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 0x7fc51006ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 >> v1:172.21.15.138:6789/0 conn(0x7fc51008bb30 legacy=0x7fc51006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc510071cf0 con 0x7fc51006cfa0 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fc5080016a0 con 0x7fc51006cfa0 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 >> v1:172.21.15.33:6789/0 conn(0x7fc51006cfa0 legacy=0x7fc51016adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4fc005c00 0x7fc4fc006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4fc006a20 0x7fc4fc006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 0x7fc4fc007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5100714e0 con 0x7fc4fc006a20 2024-04-25T01:08:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc00dd10 con 0x7fc4fc005c00 2024-04-25T01:08:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.169+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4fc00df10 con 0x7fc5100b2850 2024-04-25T01:08:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc51573f640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4fc006a20 0x7fc4fc006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc516741640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 0x7fc4fc007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 Processor -- start 2024-04-25T01:08:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 -- start start 2024-04-25T01:08:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc51573f640 1 -- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 msgr2=0x7fc4fc007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc51573f640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 0x7fc4fc007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc51573f640 1 -- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4fc005c00 msgr2=0x7fc4fc006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc515f40640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4fc005c00 0x7fc4fc006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef29424640 1 Processor -- start 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef29424640 1 -- start start 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef29424640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 0x7fef24163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef29424640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef2406bb30 con 0x7fef241624a0 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef29424640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef240b4c50 con 0x7fef2406b210 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef29424640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fef24162aa0 con 0x7fef2406b6c0 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef22ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fef241624a0 0x7fef240b4200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33906/0 (socket says 172.21.15.33:33906) 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef22ffd640 1 -- 172.21.15.33:0/1302615269 learned_addr learned my addr 172.21.15.33:0/1302615269 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef237fe640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 0x7fef24163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312895753 0 0) 0x7fef2406bb30 con 0x7fef241624a0 2024-04-25T01:08:44.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef0c003530 con 0x7fef241624a0 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc51573f640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4fc005c00 0x7fc4fc006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc51573f640 1 -- 172.21.15.33:0/3896286340 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc510071cf0 con 0x7fc4fc006a20 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc516741640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc5100b2850 0x7fc4fc007420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fc515f40640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4fc005c00 0x7fc4fc006020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48cc06d8e0 0x7f48cc14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48cc0b2d90 0x7f48cc15c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc159da0 0x7f48cc15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48cc0b2d90 0x7f48cc15c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48cc0b2d90 0x7f48cc15c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60714/0 (socket says 172.21.15.33:60714) 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48cad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc159da0 0x7f48cc15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48cad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc159da0 0x7f48cc15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50016/0 (socket says 172.21.15.33:50016) 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48cc06d8e0 0x7f48cc14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48ca59c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48cc06d8e0 0x7f48cc14ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52216/0 (socket says 172.21.15.33:52216) 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc168f70 con 0x7f48cc159da0 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48cc072040 con 0x7f48cc06d8e0 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48d0e5d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48cc1690f0 con 0x7f48cc0b2d90 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 -- 172.21.15.33:0/1506604883 learned_addr learned my addr 172.21.15.33:0/1506604883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 -- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48cc06d8e0 msgr2=0x7f48cc14ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48cc06d8e0 0x7f48cc14ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 -- 172.21.15.33:0/1506604883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc159da0 msgr2=0x7f48cc15cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc159da0 0x7f48cc15cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f48c9d9b640 1 -- 172.21.15.33:0/1506604883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48cc15dd80 con 0x7f48cc0b2d90 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90b55d2640 1 Processor -- start 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90b55d2640 1 -- start start 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90b55d2640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90b006b920 0x7f90b00b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90b55d2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162420 0x7f90b00b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90ae7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90b006b920 0x7f90b00b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90ae7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f90b006b920 0x7f90b00b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50040/0 (socket says 172.21.15.33:50040) 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90ae7fc640 1 -- 172.21.15.33:0/2649193600 learned_addr learned my addr 172.21.15.33:0/2649193600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b006dbb0 con 0x7f90b0162040 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f90b01661a0 con 0x7f90b006b920 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f90b0166480 con 0x7f90b0162420 2024-04-25T01:08:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f90af7fe640 1 --2- 172.21.15.33:0/2649193600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162420 0x7f90b00b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f9fb0640 1 Processor -- start 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f9fb0640 1 -- start start 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f9fb0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f32f40b2600 0x7f32f40af620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f9fb0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f415ab40 con 0x7f32f415a760 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f9fb0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32f40cdbc0 con 0x7f32f40cd6e0 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f9fb0640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f32f4067660 con 0x7f32f40b2600 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f2ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f32f40b2600 0x7f32f40af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f2ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f32f40b2600 0x7f32f40af620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50032/0 (socket says 172.21.15.33:50032) 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f2ffd640 1 -- 172.21.15.33:0/1584845409 learned_addr learned my addr 172.21.15.33:0/1584845409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f27fc640 1 -- 172.21.15.33:0/1584845409 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840021793 0 0) 0x7f32f40cdbc0 con 0x7f32f40cd6e0 2024-04-25T01:08:44.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f27fc640 1 -- 172.21.15.33:0/1584845409 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32e8003530 con 0x7f32f40cd6e0 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f2ffd640 1 -- 172.21.15.33:0/1584845409 >> v1:172.21.15.138:6789/0 conn(0x7f32f40cd6e0 legacy=0x7f32f4066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f2ffd640 1 -- 172.21.15.33:0/1584845409 >> v1:172.21.15.33:6789/0 conn(0x7f32f415a760 legacy=0x7f32f40b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7f32f2ffd640 1 -- 172.21.15.33:0/1584845409 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f40ad370 con 0x7f32f40b2600 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1124163489 0 0) 0x7fef0c003530 con 0x7fef241624a0 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 msgr2=0x7fef24163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef21ffb640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 0x7fef24163640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.173+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 >> v1:172.21.15.138:6789/0 conn(0x7fef2406b210 legacy=0x7fef24162f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef240b4f30 con 0x7fef241624a0 2024-04-25T01:08:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f90af7fe640 1 -- 172.21.15.33:0/2649193600 >> v2:172.21.15.33:3300/0 conn(0x7f90b006b920 msgr2=0x7f90b00b3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f90af7fe640 1 --2- 172.21.15.33:0/2649193600 >> v2:172.21.15.33:3300/0 conn(0x7f90b006b920 0x7f90b00b3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbed92640 1 Processor -- start 2024-04-25T01:08:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbed92640 1 -- start start 2024-04-25T01:08:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f90af7fe640 1 -- 172.21.15.33:0/2649193600 >> v1:172.21.15.138:6789/0 conn(0x7f90b0162040 legacy=0x7f90b006b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f90af7fe640 1 -- 172.21.15.33:0/2649193600 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b0166760 con 0x7f90b0162420 2024-04-25T01:08:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fef14001680 con 0x7fef241624a0 2024-04-25T01:08:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 >> v1:172.21.15.33:6789/0 conn(0x7fef241624a0 legacy=0x7fef240b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c0039f0 0x7fef0c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 0x7fef0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c004d50 0x7fef0c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef24162aa0 con 0x7fef0c0039f0 2024-04-25T01:08:44.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c00ba80 con 0x7fef0c004d50 2024-04-25T01:08:44.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c00bc80 con 0x7fef2406b6c0 2024-04-25T01:08:44.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc51573f640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4fc006a20 0x7fc4fc006ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc50000a9c0 tx=0x7fc50001ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ddf84f4a3e40bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef227fc640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 0x7fef0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef237fe640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c004d50 0x7fef0c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5000ab070 con 0x7fc4fc006a20 2024-04-25T01:08:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef22ffd640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c0039f0 0x7fef0c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f48c9d9b640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48cc0b2d90 0x7f48cc15c690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f48c401a370 tx=0x7f48c40194e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbed92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb80aeec0 0x7f3fb80711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbed92640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fb806d400 0x7f3fb815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f48baffd640 1 -- 172.21.15.33:0/1506604883 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40ac040 con 0x7f48cc0b2d90 2024-04-25T01:08:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f48baffd640 1 -- 172.21.15.33:0/1506604883 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48c40bc650 con 0x7f48cc0b2d90 2024-04-25T01:08:44.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f48baffd640 1 -- 172.21.15.33:0/1506604883 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4009510 con 0x7f48cc0b2d90 2024-04-25T01:08:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5000a5d00 con 0x7fc4fc006a20 2024-04-25T01:08:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc514f3e640 1 -- 172.21.15.33:0/3896286340 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50000b040 con 0x7fc4fc006a20 2024-04-25T01:08:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc5181a4640 1 -- 172.21.15.33:0/3896286340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4fc006a20 msgr2=0x7fc4fc006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc5181a4640 1 --2- 172.21.15.33:0/3896286340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4fc006a20 0x7fc4fc006ea0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc50000a9c0 tx=0x7fc50001ce00 comp rx=0 tx=0).stop 2024-04-25T01:08:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc516741640 1 -- 172.21.15.33:0/3896286340 reap_dead start 2024-04-25T01:08:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc5181a4640 1 -- 172.21.15.33:0/3896286340 shutdown_connections 2024-04-25T01:08:44.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc5181a4640 1 -- 172.21.15.33:0/3896286340 >> 172.21.15.33:0/3896286340 conn(0x7fc51006ba60 msgr2=0x7fc5100b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef237fe640 1 -- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 msgr2=0x7fef0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef237fe640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2406b6c0 0x7fef0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef237fe640 1 -- 172.21.15.33:0/1302615269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c0039f0 msgr2=0x7fef0c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef237fe640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c0039f0 0x7fef0c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fef237fe640 1 -- 172.21.15.33:0/1302615269 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c003530 con 0x7fef0c004d50 2024-04-25T01:08:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f32f2ffd640 1 --2- 172.21.15.33:0/1584845409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40b2600 0x7f32f40af620 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009870 tx=0x7f32dc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9538928d155ae85d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc5181a4640 1 -- 172.21.15.33:0/3896286340 shutdown_connections 2024-04-25T01:08:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbd58f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb80aeec0 0x7f3fb80711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbe591640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fb806d400 0x7f3fb815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbd58f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb80aeec0 0x7f3fb80711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60754/0 (socket says 172.21.15.33:60754) 2024-04-25T01:08:44.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f48d0e5d640 1 -- 172.21.15.33:0/1506604883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48cc15d200 con 0x7f48cc0b2d90 2024-04-25T01:08:44.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f48d0e5d640 1 -- 172.21.15.33:0/1506604883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48cc15d7c0 con 0x7f48cc0b2d90 2024-04-25T01:08:44.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbd58f640 1 -- 172.21.15.33:0/1501834396 learned_addr learned my addr 172.21.15.33:0/1501834396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbe591640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3fb806d400 0x7f3fb815ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50054/0 (socket says 172.21.15.33:50054) 2024-04-25T01:08:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3fb806d890 con 0x7f3fb81577c0 2024-04-25T01:08:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f90af7fe640 1 --2- 172.21.15.33:0/2649193600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162420 0x7f90b00b4590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009d90 tx=0x7f90a801c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce9c1e7f9cbf73c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f3fb80b3110 con 0x7f3fb806d400 2024-04-25T01:08:44.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb806db10 con 0x7f3fb80aeec0 2024-04-25T01:08:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f90adffb640 1 -- 172.21.15.33:0/2649193600 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80b2070 con 0x7f90b0162420 2024-04-25T01:08:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f90adffb640 1 -- 172.21.15.33:0/2649193600 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a80a58c0 con 0x7f90b0162420 2024-04-25T01:08:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90adffb640 1 -- 172.21.15.33:0/2649193600 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a5c90 con 0x7f90b0162420 2024-04-25T01:08:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f3fbe591640 1 -- 172.21.15.33:0/1501834396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb80aeec0 msgr2=0x7f3fb80711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f32f27fc640 1 -- 172.21.15.33:0/1584845409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b0070 con 0x7f32f40b2600 2024-04-25T01:08:44.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f3fbe591640 1 --2- 172.21.15.33:0/1501834396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb80aeec0 0x7f3fb80711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f32f27fc640 1 -- 172.21.15.33:0/1584845409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc0ab600 con 0x7f32f40b2600 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7f32f27fc640 1 -- 172.21.15.33:0/1584845409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0ab9d0 con 0x7f32f40b2600 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- 172.21.15.33:0/1584845409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40b2600 msgr2=0x7f32f40af620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/1584845409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40b2600 0x7f32f40af620 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009870 tx=0x7f32dc01c490 comp rx=0 tx=0).stop 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- 172.21.15.33:0/1584845409 shutdown_connections 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/1584845409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40b2600 0x7f32f40af620 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- 172.21.15.33:0/1584845409 >> 172.21.15.33:0/1584845409 conn(0x7f32f40b50c0 msgr2=0x7f32f40b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f3fbe591640 1 -- 172.21.15.33:0/1501834396 >> v1:172.21.15.33:6789/0 conn(0x7f3fb81577c0 legacy=0x7f3fb80ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f3fbcd8e640 1 -- 172.21.15.33:0/1501834396 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4065033057 0 0) 0x7f3fb4001120 con 0x7f3fb81577c0 2024-04-25T01:08:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- 172.21.15.33:0/1584845409 shutdown_connections 2024-04-25T01:08:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f3fbe591640 1 -- 172.21.15.33:0/1501834396 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb80b27e0 con 0x7f3fb806d400 2024-04-25T01:08:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- 172.21.15.33:0/1584845409 wait complete. 2024-04-25T01:08:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162420 msgr2=0x7f90b00b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2649193600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162420 0x7f90b00b4590 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009d90 tx=0x7f90a801c930 comp rx=0 tx=0).stop 2024-04-25T01:08:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 shutdown_connections 2024-04-25T01:08:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2649193600 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162420 0x7f90b00b4590 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2649193600 >> v2:172.21.15.33:3300/0 conn(0x7f90b006b920 0x7f90b00b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 >> 172.21.15.33:0/2649193600 conn(0x7f90b006cb10 msgr2=0x7f90b00b28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.177+0000 7fc5181a4640 1 -- 172.21.15.33:0/3896286340 wait complete. 2024-04-25T01:08:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 Processor -- start 2024-04-25T01:08:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 shutdown_connections 2024-04-25T01:08:44.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 Processor -- start 2024-04-25T01:08:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f48baffd640 1 -- 172.21.15.33:0/1506604883 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f48c40af070 con 0x7f48cc0b2d90 2024-04-25T01:08:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f48baffd640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f48ac0678c0 0x7f48ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f48baffd640 1 -- 172.21.15.33:0/1506604883 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48c4120510 con 0x7f48cc0b2d90 2024-04-25T01:08:44.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 Processor -- start 2024-04-25T01:08:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 -- start start 2024-04-25T01:08:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5100ade40 0x7fc5101645e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc510162bd0 0x7fc5100cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5100ce0c0 0x7fc5100a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fef237fe640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c004d50 0x7fef0c0051d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009f90 tx=0x7fef1c01cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43efe4c765a0167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- 172.21.15.33:0/2649193600 wait complete. 2024-04-25T01:08:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 Processor -- start 2024-04-25T01:08:44.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc516741640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5100ce0c0 0x7fc5100a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc516741640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5100ce0c0 0x7fc5100a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50064/0 (socket says 172.21.15.33:50064) 2024-04-25T01:08:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc515f40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5100ade40 0x7fc5101645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc515f40640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5100ade40 0x7fc5101645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52238/0 (socket says 172.21.15.33:52238) 2024-04-25T01:08:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c0b2070 con 0x7fef0c004d50 2024-04-25T01:08:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1c0a5b10 con 0x7fef0c004d50 2024-04-25T01:08:44.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fef21ffb640 1 -- 172.21.15.33:0/1302615269 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c009c10 con 0x7fef0c004d50 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc516741640 1 -- 172.21.15.33:0/1763890592 learned_addr learned my addr 172.21.15.33:0/1763890592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc51016bf20 con 0x7fc5100ce0c0 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc51006d410 con 0x7fc5100ade40 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc51016c0a0 con 0x7fc510162bd0 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc51573f640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc510162bd0 0x7fc5100cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f48d0e5d640 1 -- 172.21.15.33:0/1506604883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48cc04f580 con 0x7f48cc0b2d90 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 -- start start 2024-04-25T01:08:44.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c15c6a0 0x7f179c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f179c0ae690 0x7f179c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f179c0b31d0 0x7f179c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f179c0af030 con 0x7f179c0ae690 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f179c0b4150 con 0x7f179c0b31d0 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179bfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f179c06e080 con 0x7f179c15c6a0 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- start start 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90b006b920 0x7f90b009a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162040 0x7f90b02034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0162420 0x7f90b02039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00b52e0 con 0x7f90b006b920 2024-04-25T01:08:44.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00b05b0 con 0x7f90b0162420 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f90b55d2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00b5460 con 0x7f90b0162040 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- start start 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f40b2600 0x7f32f41557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40cd6e0 0x7f32f42036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32f415a760 0x7f32f41570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4163bd0 con 0x7f32f40cd6e0 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f32f4163a50 con 0x7f32f40b2600 2024-04-25T01:08:44.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f32f9fb0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f32f4163d50 con 0x7f32f415a760 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f179c0ae690 0x7f179c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f179c0ae690 0x7f179c0aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50076/0 (socket says 172.21.15.33:50076) 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 -- 172.21.15.33:0/1820156093 learned_addr learned my addr 172.21.15.33:0/1820156093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 -- 172.21.15.33:0/1820156093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c15c6a0 msgr2=0x7f179c06e510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 --2- 172.21.15.33:0/1820156093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c15c6a0 0x7f179c06e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 -- 172.21.15.33:0/1820156093 >> v2:172.21.15.138:3300/0 conn(0x7f179c0b31d0 msgr2=0x7f179c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 --2- 172.21.15.33:0/1820156093 >> v2:172.21.15.138:3300/0 conn(0x7f179c0b31d0 0x7f179c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f179a7fc640 1 -- 172.21.15.33:0/1820156093 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c0b3ba0 con 0x7f179c0ae690 2024-04-25T01:08:44.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7f48ca59c640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f48ac0678c0 0x7f48ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32f415a760 0x7f32f41570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 -- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c004d50 msgr2=0x7fef0c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 --2- 172.21.15.33:0/1302615269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c004d50 0x7fef0c0051d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009f90 tx=0x7fef1c01cc70 comp rx=0 tx=0).stop 2024-04-25T01:08:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc515f40640 1 -- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc510162bd0 msgr2=0x7fc5100cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc515f40640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc510162bd0 0x7fc5100cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc515f40640 1 -- 172.21.15.33:0/1763890592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5100ce0c0 msgr2=0x7fc5100a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc515f40640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5100ce0c0 0x7fc5100a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.181+0000 7fc515f40640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc510164b20 con 0x7fc5100ade40 2024-04-25T01:08:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40cd6e0 0x7f32f42036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32f415a760 0x7f32f41570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60784/0 (socket says 172.21.15.33:60784) 2024-04-25T01:08:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f3fff640 1 -- 172.21.15.33:0/2763340883 learned_addr learned my addr 172.21.15.33:0/2763340883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90ae7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162040 0x7f90b02034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90ae7fc640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162040 0x7f90b02034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60798/0 (socket says 172.21.15.33:60798) 2024-04-25T01:08:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90ae7fc640 1 -- 172.21.15.33:0/2963229721 learned_addr learned my addr 172.21.15.33:0/2963229721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90af7fe640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0162420 0x7f90b02039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90b006b920 0x7f90b009a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef237fe640 1 -- 172.21.15.33:0/1302615269 reap_dead start 2024-04-25T01:08:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 -- 172.21.15.33:0/1302615269 shutdown_connections 2024-04-25T01:08:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 -- 172.21.15.33:0/1302615269 >> 172.21.15.33:0/1302615269 conn(0x7fef240b27b0 msgr2=0x7fef240b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f48ca59c640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f48ac0678c0 0x7f48ac069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48bc007fd0 tx=0x7f48bc007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f2ffd640 1 -- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32f415a760 msgr2=0x7f32f41570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f2ffd640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32f415a760 0x7f32f41570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f2ffd640 1 -- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f40b2600 msgr2=0x7f32f41557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f2ffd640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f40b2600 0x7f32f41557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f2ffd640 1 -- 172.21.15.33:0/2763340883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f4156040 con 0x7f32f40cd6e0 2024-04-25T01:08:44.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fc515f40640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5100ade40 0x7fc5101645e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00d500 tx=0x7fc50c00d9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90aeffd640 1 -- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162040 msgr2=0x7f90b02034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90aeffd640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162040 0x7f90b02034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90aeffd640 1 -- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0162420 msgr2=0x7f90b02039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90aeffd640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0162420 0x7f90b02039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90aeffd640 1 -- 172.21.15.33:0/2963229721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b01688c0 con 0x7f90b006b920 2024-04-25T01:08:44.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 -- 172.21.15.33:0/1302615269 shutdown_connections 2024-04-25T01:08:44.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32f2ffd640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40cd6e0 0x7f32f42036b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32f406fda0 tx=0x7f32dc019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 -- 172.21.15.33:0/1302615269 wait complete. 2024-04-25T01:08:44.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbe591640 1 --2- 172.21.15.33:0/1501834396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb806d400 0x7f3fb815ad20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000c480 tx=0x7f3fb001d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43aed1df85ef5521 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90aeffd640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90b006b920 0x7f90b009a9d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90a000c360 tx=0x7f90a000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90977fe640 1 -- 172.21.15.33:0/2963229721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0017070 con 0x7f90b006b920 2024-04-25T01:08:44.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbcd8e640 1 -- 172.21.15.33:0/1501834396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00b2070 con 0x7f3fb806d400 2024-04-25T01:08:44.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f32e3fff640 1 -- 172.21.15.33:0/2763340883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b0050 con 0x7f32f40cd6e0 2024-04-25T01:08:44.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b009b3b0 con 0x7f90b006b920 2024-04-25T01:08:44.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbcd8e640 1 -- 172.21.15.33:0/1501834396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb00ad440 con 0x7f3fb806d400 2024-04-25T01:08:44.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbcd8e640 1 -- 172.21.15.33:0/1501834396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00ad810 con 0x7f3fb806d400 2024-04-25T01:08:44.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fc5067fc640 1 -- 172.21.15.33:0/1763890592 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003620 con 0x7fc5100ade40 2024-04-25T01:08:44.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fc5067fc640 1 -- 172.21.15.33:0/1763890592 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50c017070 con 0x7fc5100ade40 2024-04-25T01:08:44.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b009b970 con 0x7f90b006b920 2024-04-25T01:08:44.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90977fe640 1 -- 172.21.15.33:0/2963229721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a0003600 con 0x7f90b006b920 2024-04-25T01:08:44.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90977fe640 1 -- 172.21.15.33:0/2963229721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0003a30 con 0x7f90b006b920 2024-04-25T01:08:44.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb806d400 msgr2=0x7f3fb815ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbed92640 1 --2- 172.21.15.33:0/1501834396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb806d400 0x7f3fb815ad20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000c480 tx=0x7f3fb001d030 comp rx=0 tx=0).stop 2024-04-25T01:08:44.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fc5067fc640 1 -- 172.21.15.33:0/1763890592 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0039e0 con 0x7fc5100ade40 2024-04-25T01:08:44.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc510164da0 con 0x7fc5100ade40 2024-04-25T01:08:44.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc510165360 con 0x7fc5100ade40 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 shutdown_connections 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbed92640 1 --2- 172.21.15.33:0/1501834396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb806d400 0x7f3fb815ad20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbed92640 1 --2- 172.21.15.33:0/1501834396 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb80aeec0 0x7f3fb80711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 >> 172.21.15.33:0/1501834396 conn(0x7f3fb80b14e0 msgr2=0x7f3fb80b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 Processor -- start 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 -- start start 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef241593d0 0x7fef24166e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef241fae00 0x7fef2415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7fef29424640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2416b9b0 0x7fef2415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef29424640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef240b3dc0 con 0x7fef241593d0 2024-04-25T01:08:44.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef29424640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef240b3c40 con 0x7fef241fae00 2024-04-25T01:08:44.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef29424640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fef24163e50 con 0x7fef2416b9b0 2024-04-25T01:08:44.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.185+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90900052a0 con 0x7f90b006b920 2024-04-25T01:08:44.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 shutdown_connections 2024-04-25T01:08:44.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f4156320 con 0x7f32f40cd6e0 2024-04-25T01:08:44.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f41568e0 con 0x7f32f40cd6e0 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef241593d0 0x7fef24166e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef237fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2416b9b0 0x7fef2415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef241593d0 0x7fef24166e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50100/0 (socket says 172.21.15.33:50100) 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 -- 172.21.15.33:0/967313327 learned_addr learned my addr 172.21.15.33:0/967313327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef227fc640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef241fae00 0x7fef2415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 -- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2416b9b0 msgr2=0x7fef2415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2416b9b0 0x7fef2415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 -- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef241fae00 msgr2=0x7fef2415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef241fae00 0x7fef2415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 -- 172.21.15.33:0/967313327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef24167700 con 0x7fef241593d0 2024-04-25T01:08:44.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fef22ffd640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef241593d0 0x7fef24166e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef140072f0 tx=0x7fef14004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f4069040 con 0x7f32f40cd6e0 2024-04-25T01:08:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f90977fe640 1 -- 172.21.15.33:0/2963229721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f90a001a070 con 0x7f90b006b920 2024-04-25T01:08:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f179a7fc640 1 --2- 172.21.15.33:0/1820156093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0ae690 0x7f179c0aeaf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f179c165c30 tx=0x7f178c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294c1bd9e24148ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f90977fe640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f909c067910 0x7f909c069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 -- 172.21.15.33:0/1501834396 wait complete. 2024-04-25T01:08:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 Processor -- start 2024-04-25T01:08:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f32e3fff640 1 -- 172.21.15.33:0/2763340883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32dc0ba070 con 0x7f32f40cd6e0 2024-04-25T01:08:44.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f32e3fff640 1 -- 172.21.15.33:0/2763340883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b4810 con 0x7f32f40cd6e0 2024-04-25T01:08:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a62c7640 1 Processor -- start 2024-04-25T01:08:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f90ae7fc640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f909c067910 0x7f909c069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a62c7640 1 -- start start 2024-04-25T01:08:44.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f90ae7fc640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f909c067910 0x7f909c069dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90b016bc00 tx=0x7f90a4009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a62c7640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb598034460 0x7fb598008090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a62c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb598034840 0x7fb5980381b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a62c7640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5980389a0 con 0x7fb59804f9b0 2024-04-25T01:08:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a62c7640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb5980073c0 con 0x7fb598034460 2024-04-25T01:08:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a62c7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb5980386f0 con 0x7fb598034840 2024-04-25T01:08:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a5ac6640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb59804f9b0 0x7fb5980e4a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33936/0 (socket says 172.21.15.33:33936) 2024-04-25T01:08:44.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a5ac6640 1 -- 172.21.15.33:0/3313589645 learned_addr learned my addr 172.21.15.33:0/3313589645 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f80052a0 con 0x7fc5100ade40 2024-04-25T01:08:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fc5067fc640 1 -- 172.21.15.33:0/1763890592 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc50c003b80 con 0x7fc5100ade40 2024-04-25T01:08:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fc5067fc640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4f4067990 0x7fc4f4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fc5067fc640 1 -- 172.21.15.33:0/1763890592 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc50c085f70 con 0x7fc5100ade40 2024-04-25T01:08:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fc51573f640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4f4067990 0x7fc4f4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fc51573f640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4f4067990 0x7fc4f4069e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc5000b0000 tx=0x7fc5000a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f90977fe640 1 -- 172.21.15.33:0/2963229721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f90a00539d0 con 0x7f90b006b920 2024-04-25T01:08:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a52c5640 1 --2- 172.21.15.33:0/3313589645 >> v2:172.21.15.33:3300/0 conn(0x7fb598034460 0x7fb598008090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7fb5a4ac4640 1 --2- 172.21.15.33:0/3313589645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb598034840 0x7fb5980381b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 -- start start 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fb806d400 0x7f3fb8142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb80aeec0 0x7f3fb80cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb81577c0 0x7f3fb80ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb80b3f30 con 0x7f3fb80aeec0 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb80b3db0 con 0x7f3fb806d400 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbed92640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3fb80b40b0 con 0x7f3fb81577c0 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbd58f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb80aeec0 0x7f3fb80cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbd58f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb80aeec0 0x7f3fb80cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50128/0 (socket says 172.21.15.33:50128) 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbd58f640 1 -- 172.21.15.33:0/463824703 learned_addr learned my addr 172.21.15.33:0/463824703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.189+0000 7f3fbe591640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb81577c0 0x7f3fb80ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f3fbdd90640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fb806d400 0x7f3fb8142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f3fbd58f640 1 -- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb81577c0 msgr2=0x7f3fb80ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f3fbd58f640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb81577c0 0x7f3fb80ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f3fbd58f640 1 -- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fb806d400 msgr2=0x7f3fb8142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f3fbd58f640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fb806d400 0x7f3fb8142340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f3fbd58f640 1 -- 172.21.15.33:0/463824703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb80ae2b0 con 0x7f3fb80aeec0 2024-04-25T01:08:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7fb5a52c5640 1 -- 172.21.15.33:0/3313589645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb598034840 msgr2=0x7fb5980381b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7fb5a52c5640 1 --2- 172.21.15.33:0/3313589645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb598034840 0x7fb5980381b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7fb5a52c5640 1 -- 172.21.15.33:0/3313589645 >> v1:172.21.15.33:6789/0 conn(0x7fb59804f9b0 legacy=0x7fb5980e4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7fb5a52c5640 1 -- 172.21.15.33:0/3313589645 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb598006e00 con 0x7fb598034460 2024-04-25T01:08:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f3fbd58f640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb80aeec0 0x7f3fb80cef50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e930 tx=0x7f3fa800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7fb5a52c5640 1 --2- 172.21.15.33:0/3313589645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb598008090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0097c0 tx=0x7fb59c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18997761a49dcff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.193+0000 7f90977fe640 1 -- 172.21.15.33:0/2963229721 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90a0021020 con 0x7f90b006b920 2024-04-25T01:08:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb597fff640 1 -- 172.21.15.33:0/3313589645 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0ab600 con 0x7fb598034460 2024-04-25T01:08:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fef0b7fe640 1 -- 172.21.15.33:0/967313327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1400f9c0 con 0x7fef241593d0 2024-04-25T01:08:44.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f1799ffb640 1 -- 172.21.15.33:0/1820156093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0b2070 con 0x7f179c0ae690 2024-04-25T01:08:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef241679e0 con 0x7fef241593d0 2024-04-25T01:08:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef24167360 con 0x7fef241593d0 2024-04-25T01:08:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fef0b7fe640 1 -- 172.21.15.33:0/967313327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef14018070 con 0x7fef241593d0 2024-04-25T01:08:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fef0b7fe640 1 -- 172.21.15.33:0/967313327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1400fd60 con 0x7fef241593d0 2024-04-25T01:08:44.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb597fff640 1 -- 172.21.15.33:0/3313589645 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c0b0070 con 0x7fb598034460 2024-04-25T01:08:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb597fff640 1 -- 172.21.15.33:0/3313589645 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0ab9c0 con 0x7fb598034460 2024-04-25T01:08:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef040052a0 con 0x7fef241593d0 2024-04-25T01:08:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f32e3fff640 1 -- 172.21.15.33:0/2763340883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f32dc0b49b0 con 0x7f32f40cd6e0 2024-04-25T01:08:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f32e3fff640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f32d4067990 0x7f32d4069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f32e3fff640 1 -- 172.21.15.33:0/2763340883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f32dc120a10 con 0x7f32f40cd6e0 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f1799ffb640 1 -- 172.21.15.33:0/1820156093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c0a9070 con 0x7f179c0ae690 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 -- 172.21.15.33:0/1820156093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0ae690 msgr2=0x7f179c0aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 --2- 172.21.15.33:0/1820156093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0ae690 0x7f179c0aeaf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f179c165c30 tx=0x7f178c01c4b0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 -- 172.21.15.33:0/1820156093 shutdown_connections 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 --2- 172.21.15.33:0/1820156093 >> v2:172.21.15.138:3300/0 conn(0x7f179c0b31d0 0x7f179c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 --2- 172.21.15.33:0/1820156093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0ae690 0x7f179c0aeaf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 --2- 172.21.15.33:0/1820156093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c15c6a0 0x7f179c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 -- 172.21.15.33:0/1820156093 >> 172.21.15.33:0/1820156093 conn(0x7f179c06b3e0 msgr2=0x7f179c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb5a62c7640 1 -- 172.21.15.33:0/3313589645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 msgr2=0x7fb598008090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/3313589645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb598008090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0097c0 tx=0x7fb59c01c3e0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb5a62c7640 1 -- 172.21.15.33:0/3313589645 shutdown_connections 2024-04-25T01:08:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/3313589645 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb598034840 0x7fb5980381b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/3313589645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb598008090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7fb5a62c7640 1 -- 172.21.15.33:0/3313589645 >> 172.21.15.33:0/3313589645 conn(0x7fb598055360 msgr2=0x7fb598055770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 -- 172.21.15.33:0/1820156093 shutdown_connections 2024-04-25T01:08:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f48baffd640 1 -- 172.21.15.33:0/1506604883 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f48c40c3030 con 0x7f48cc0b2d90 2024-04-25T01:08:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f179bfff640 1 -- 172.21.15.33:0/1820156093 wait complete. 2024-04-25T01:08:44.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f3fae7fc640 1 -- 172.21.15.33:0/463824703 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa8017070 con 0x7f3fb80aeec0 2024-04-25T01:08:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f3fae7fc640 1 -- 172.21.15.33:0/463824703 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fa800ca00 con 0x7f3fb80aeec0 2024-04-25T01:08:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.197+0000 7f3fae7fc640 1 -- 172.21.15.33:0/463824703 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa800cdf0 con 0x7f3fb80aeec0 2024-04-25T01:08:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 Processor -- start 2024-04-25T01:08:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb80adcf0 con 0x7f3fb80aeec0 2024-04-25T01:08:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb8167b70 con 0x7f3fb80aeec0 2024-04-25T01:08:44.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb804f580 con 0x7f3fb80aeec0 2024-04-25T01:08:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7fb5a62c7640 1 -- 172.21.15.33:0/3313589645 shutdown_connections 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 -- start start 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c0ae690 0x7f179c141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0b31d0 0x7f179c145a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c15c6a0 0x7f179c145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f179c071270 con 0x7f179c0b31d0 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f179c15ca80 con 0x7f179c15c6a0 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179bfff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f179c0713f0 con 0x7f179c0ae690 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0b31d0 0x7f179c145a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0b31d0 0x7f179c145a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50142/0 (socket says 172.21.15.33:50142) 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 -- 172.21.15.33:0/1112489630 learned_addr learned my addr 172.21.15.33:0/1112489630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179affd640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c0ae690 0x7f179c141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179b7fe640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c15c6a0 0x7f179c145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 -- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c0ae690 msgr2=0x7f179c141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c0ae690 0x7f179c141cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 -- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c15c6a0 msgr2=0x7f179c145fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c15c6a0 0x7f179c145fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 -- 172.21.15.33:0/1112489630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f179c152330 con 0x7f179c0b31d0 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179affd640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c0ae690 0x7f179c141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179b7fe640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c15c6a0 0x7f179c145fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:44.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f179a7fc640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0b31d0 0x7f179c145a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f178c0b0040 tx=0x7f178c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7fb5a62c7640 1 -- 172.21.15.33:0/3313589645 wait complete. 2024-04-25T01:08:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7fb5a62c7640 1 Processor -- start 2024-04-25T01:08:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7fb5a62c7640 1 -- start start 2024-04-25T01:08:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7f32f37fe640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f32d4067990 0x7f32d4069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.201+0000 7fc5067fc640 1 -- 172.21.15.33:0/1763890592 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc510164b20 con 0x7fc5100ade40 2024-04-25T01:08:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.205+0000 7f32f37fe640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f32d4067990 0x7f32d4069e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f32f416d750 tx=0x7f32e4006bb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.205+0000 7f32e3fff640 1 -- 172.21.15.33:0/2763340883 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f32dc016110 con 0x7f32f40cd6e0 2024-04-25T01:08:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a62c7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb59804c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a62c7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb598034840 0x7fb59804cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a62c7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59804f9b0 0x7fb5980e3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a62c7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb59803b0a0 con 0x7fb598034460 2024-04-25T01:08:44.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a62c7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb59803af20 con 0x7fb598034840 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a62c7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb59803b220 con 0x7fb59804f9b0 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb59804c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb59804c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50146/0 (socket says 172.21.15.33:50146) 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 -- 172.21.15.33:0/1667086309 learned_addr learned my addr 172.21.15.33:0/1667086309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a5ac6640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59804f9b0 0x7fb5980e3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a4ac4640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb598034840 0x7fb59804cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 -- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59804f9b0 msgr2=0x7fb5980e3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59804f9b0 0x7fb5980e3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 -- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb598034840 msgr2=0x7fb59804cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb598034840 0x7fb59804cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 -- 172.21.15.33:0/1667086309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5980e0450 con 0x7fb598034460 2024-04-25T01:08:44.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.209+0000 7fb5a52c5640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb59804c430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0ad040 tx=0x7fb59c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fef0b7fe640 1 -- 172.21.15.33:0/967313327 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fef1401d460 con 0x7fef241593d0 2024-04-25T01:08:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fef0b7fe640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fef100678c0 0x7fef10069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fb595ffb640 1 -- 172.21.15.33:0/1667086309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0ab9c0 con 0x7fb598034460 2024-04-25T01:08:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fef227fc640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fef100678c0 0x7fef10069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5980eb9f0 con 0x7fb598034460 2024-04-25T01:08:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59804d6d0 con 0x7fb598034460 2024-04-25T01:08:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7f3fae7fc640 1 -- 172.21.15.33:0/463824703 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3fa801a070 con 0x7f3fb80aeec0 2024-04-25T01:08:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7f3fae7fc640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f9c0678f0 0x7f3f9c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7f17837fe640 1 -- 172.21.15.33:0/1112489630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0ba020 con 0x7f179c0b31d0 2024-04-25T01:08:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fb595ffb640 1 -- 172.21.15.33:0/1667086309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb59c0b0020 con 0x7fb598034460 2024-04-25T01:08:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7fb595ffb640 1 -- 172.21.15.33:0/1667086309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0abe20 con 0x7fb598034460 2024-04-25T01:08:44.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.217+0000 7fef227fc640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fef100678c0 0x7fef10069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef241fb240 tx=0x7fef18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7f17837fe640 1 -- 172.21.15.33:0/1112489630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178c0ab6e0 con 0x7f179c0b31d0 2024-04-25T01:08:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7f17837fe640 1 -- 172.21.15.33:0/1112489630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178c0b2040 con 0x7f179c0b31d0 2024-04-25T01:08:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f179c1525a0 con 0x7f179c0b31d0 2024-04-25T01:08:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.213+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f179c152b60 con 0x7f179c0b31d0 2024-04-25T01:08:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.217+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f179c04f580 con 0x7f179c0b31d0 2024-04-25T01:08:44.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.217+0000 7fef0b7fe640 1 -- 172.21.15.33:0/967313327 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fef14054e70 con 0x7fef241593d0 2024-04-25T01:08:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.217+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5800052a0 con 0x7fb598034460 2024-04-25T01:08:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.217+0000 7f3fae7fc640 1 -- 172.21.15.33:0/463824703 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3fa80549e0 con 0x7f3fb80aeec0 2024-04-25T01:08:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.217+0000 7f3fbdd90640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f9c0678f0 0x7f3f9c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.217+0000 7f3fbdd90640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f9c0678f0 0x7f3f9c069db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4001040 tx=0x7f3fb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.225+0000 7fef0b7fe640 1 -- 172.21.15.33:0/967313327 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fef140574d0 con 0x7fef241593d0 2024-04-25T01:08:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.233+0000 7f3fae7fc640 1 -- 172.21.15.33:0/463824703 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3fa8057040 con 0x7f3fb80aeec0 2024-04-25T01:08:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7f17837fe640 1 -- 172.21.15.33:0/1112489630 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f178c0bd020 con 0x7f179c0b31d0 2024-04-25T01:08:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7f17837fe640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1788067990 0x7f1788069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7f17837fe640 1 -- 172.21.15.33:0/1112489630 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f178c120230 con 0x7f179c0b31d0 2024-04-25T01:08:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7fb595ffb640 1 -- 172.21.15.33:0/1667086309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb59c0c3060 con 0x7fb598034460 2024-04-25T01:08:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7fb595ffb640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb5840678c0 0x7fb584069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7fb595ffb640 1 -- 172.21.15.33:0/1667086309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb59c11fa50 con 0x7fb598034460 2024-04-25T01:08:44.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7f179affd640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1788067990 0x7f1788069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.237+0000 7fb5a4ac4640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb5840678c0 0x7fb584069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.241+0000 7fb5a4ac4640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb5840678c0 0x7fb584069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb59804cc30 tx=0x7fb590009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.241+0000 7f179affd640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1788067990 0x7f1788069e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f179000a830 tx=0x7f1790008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:44.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.245+0000 7f17837fe640 1 -- 172.21.15.33:0/1112489630 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f178c0c4030 con 0x7f179c0b31d0 2024-04-25T01:08:44.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.257+0000 7fb595ffb640 1 -- 172.21.15.33:0/1667086309 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb59c016090 con 0x7fb598034460 2024-04-25T01:08:44.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.505+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc4f8005860 con 0x7fc5100ade40 2024-04-25T01:08:44.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.505+0000 7fc5067fc640 1 -- 172.21.15.33:0/1763890592 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc50c056190 con 0x7fc5100ade40 2024-04-25T01:08:44.498 INFO:teuthology.orchestra.run.smithi033.stdout:64424509447 2024-04-25T01:08:44.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.505+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9090005860 con 0x7f90b006b920 2024-04-25T01:08:44.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.505+0000 7f90977fe640 1 -- 172.21.15.33:0/2963229721 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f90b01688c0 con 0x7f90b006b920 2024-04-25T01:08:44.499 INFO:teuthology.orchestra.run.smithi033.stdout:64424509446 2024-04-25T01:08:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4f4067990 msgr2=0x7fc4f4069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4f4067990 0x7fc4f4069e50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc5000b0000 tx=0x7fc5000a7040 comp rx=0 tx=0).stop 2024-04-25T01:08:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5100ade40 msgr2=0x7fc5101645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5100ade40 0x7fc5101645e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00d500 tx=0x7fc50c00d9c0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 shutdown_connections 2024-04-25T01:08:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4f4067990 0x7fc4f4069e50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5100ce0c0 0x7fc5100a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc510162bd0 0x7fc5100cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 --2- 172.21.15.33:0/1763890592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5100ade40 0x7fc5101645e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 >> 172.21.15.33:0/1763890592 conn(0x7fc51006ba60 msgr2=0x7fc5100b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 shutdown_connections 2024-04-25T01:08:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.509+0000 7fc5181a4640 1 -- 172.21.15.33:0/1763890592 wait complete. 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f909c067910 msgr2=0x7f909c069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f909c067910 0x7f909c069dd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90b016bc00 tx=0x7f90a4009240 comp rx=0 tx=0).stop 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90b006b920 msgr2=0x7f90b009a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90b006b920 0x7f90b009a9d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90a000c360 tx=0x7f90a000c820 comp rx=0 tx=0).stop 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 shutdown_connections 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f909c067910 0x7f909c069dd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90b0162420 0x7f90b02039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90b0162040 0x7f90b02034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 --2- 172.21.15.33:0/2963229721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90b006b920 0x7f90b009a9d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 >> 172.21.15.33:0/2963229721 conn(0x7f90b006cb10 msgr2=0x7f90b00ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 shutdown_connections 2024-04-25T01:08:44.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f90b55d2640 1 -- 172.21.15.33:0/2963229721 wait complete. 2024-04-25T01:08:44.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.517+0000 7f48d0e5d640 1 -- 172.21.15.33:0/1506604883 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f48cc0cf550 con 0x7f48cc0b2d90 2024-04-25T01:08:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.521+0000 7f48baffd640 1 -- 172.21.15.33:0/1506604883 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48cc0b3f30 con 0x7f48cc0b2d90 2024-04-25T01:08:44.515 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2024-04-25T01:08:44.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.525+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f32f4071840 con 0x7f32f40cd6e0 2024-04-25T01:08:44.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.525+0000 7f32e3fff640 1 -- 172.21.15.33:0/2763340883 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f32dc0f11f0 con 0x7f32f40cd6e0 2024-04-25T01:08:44.517 INFO:teuthology.orchestra.run.smithi033.stdout:64424509447 2024-04-25T01:08:44.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 -- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f48ac0678c0 msgr2=0x7f48ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f32d4067990 msgr2=0x7f32d4069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f32d4067990 0x7f32d4069e50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f32f416d750 tx=0x7f32e4006bb0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40cd6e0 msgr2=0x7f32f42036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f48ac0678c0 0x7f48ac069d80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f48bc007fd0 tx=0x7f48bc007e40 comp rx=0 tx=0).stop 2024-04-25T01:08:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 -- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48cc0b2d90 msgr2=0x7f48cc15c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40cd6e0 0x7f32f42036b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f32f406fda0 tx=0x7f32dc019230 comp rx=0 tx=0).stop 2024-04-25T01:08:44.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48cc0b2d90 0x7f48cc15c690 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f48c401a370 tx=0x7f48c40194e0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 shutdown_connections 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f32d4067990 0x7f32d4069e50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f32f415a760 0x7f32f41570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32f40cd6e0 0x7f32f42036b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 --2- 172.21.15.33:0/2763340883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32f40b2600 0x7f32f41557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 >> 172.21.15.33:0/2763340883 conn(0x7f32f40b50c0 msgr2=0x7f32f4149370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 -- 172.21.15.33:0/1506604883 shutdown_connections 2024-04-25T01:08:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f48ac0678c0 0x7f48ac069d80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 shutdown_connections 2024-04-25T01:08:44.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f32f9fb0640 1 -- 172.21.15.33:0/2763340883 wait complete. 2024-04-25T01:08:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f48cc159da0 0x7f48cc15cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48cc0b2d90 0x7f48cc15c690 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 --2- 172.21.15.33:0/1506604883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48cc06d8e0 0x7f48cc14ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 -- 172.21.15.33:0/1506604883 >> 172.21.15.33:0/1506604883 conn(0x7f48cc06ba60 msgr2=0x7f48cc0af550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 -- 172.21.15.33:0/1506604883 shutdown_connections 2024-04-25T01:08:44.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.529+0000 7f48b8ff9640 1 -- 172.21.15.33:0/1506604883 wait complete. 2024-04-25T01:08:44.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.3 2024-04-25T01:08:44.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509446 for osd.7 2024-04-25T01:08:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.553+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fef04005860 con 0x7fef241593d0 2024-04-25T01:08:44.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.553+0000 7fef0b7fe640 1 -- 172.21.15.33:0/967313327 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fef14056dd0 con 0x7fef241593d0 2024-04-25T01:08:44.545 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2024-04-25T01:08:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.553+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fef100678c0 msgr2=0x7fef10069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.553+0000 7fef29424640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fef100678c0 0x7fef10069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fef241fb240 tx=0x7fef18009040 comp rx=0 tx=0).stop 2024-04-25T01:08:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.553+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef241593d0 msgr2=0x7fef24166e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.553+0000 7fef29424640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef241593d0 0x7fef24166e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef140072f0 tx=0x7fef14004f90 comp rx=0 tx=0).stop 2024-04-25T01:08:44.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 shutdown_connections 2024-04-25T01:08:44.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fef100678c0 0x7fef10069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef2416b9b0 0x7fef2415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef241fae00 0x7fef2415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 --2- 172.21.15.33:0/967313327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef241593d0 0x7fef24166e20 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fef140072f0 tx=0x7fef14004f90 comp rx=0 tx=0).stop 2024-04-25T01:08:44.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 >> 172.21.15.33:0/967313327 conn(0x7fef240b27b0 msgr2=0x7fef24164190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 shutdown_connections 2024-04-25T01:08:44.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7fef29424640 1 -- 172.21.15.33:0/967313327 wait complete. 2024-04-25T01:08:44.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.6 2024-04-25T01:08:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3fb815be20 con 0x7f3fb80aeec0 2024-04-25T01:08:44.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2024-04-25T01:08:44.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.557+0000 7f3fae7fc640 1 -- 172.21.15.33:0/463824703 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3fa8056940 con 0x7f3fb80aeec0 2024-04-25T01:08:44.552 INFO:teuthology.orchestra.run.smithi033.stdout:64424509447 2024-04-25T01:08:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f9c0678f0 msgr2=0x7f3f9c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f9c0678f0 0x7f3f9c069db0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4001040 tx=0x7f3fb4009040 comp rx=0 tx=0).stop 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb80aeec0 msgr2=0x7f3fb80cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb80aeec0 0x7f3fb80cef50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3fa800e930 tx=0x7f3fa800edf0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 shutdown_connections 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3f9c0678f0 0x7f3f9c069db0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3fb81577c0 0x7f3fb80ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fb80aeec0 0x7f3fb80cef50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 --2- 172.21.15.33:0/463824703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fb806d400 0x7f3fb8142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 >> 172.21.15.33:0/463824703 conn(0x7f3fb80b14e0 msgr2=0x7f3fb80720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 shutdown_connections 2024-04-25T01:08:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.561+0000 7f3fbed92640 1 -- 172.21.15.33:0/463824703 wait complete. 2024-04-25T01:08:44.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.565+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f179c072230 con 0x7f179c0b31d0 2024-04-25T01:08:44.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.565+0000 7f17837fe640 1 -- 172.21.15.33:0/1112489630 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f178c016130 con 0x7f179c0b31d0 2024-04-25T01:08:44.559 INFO:teuthology.orchestra.run.smithi033.stdout:64424509448 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1788067990 msgr2=0x7f1788069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1788067990 0x7f1788069e50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f179000a830 tx=0x7f1790008040 comp rx=0 tx=0).stop 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0b31d0 msgr2=0x7f179c145a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0b31d0 0x7f179c145a80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f178c0b0040 tx=0x7f178c019250 comp rx=0 tx=0).stop 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 shutdown_connections 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1788067990 0x7f1788069e50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f179c15c6a0 0x7f179c145fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f179c0b31d0 0x7f179c145a80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 --2- 172.21.15.33:0/1112489630 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f179c0ae690 0x7f179c141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 >> 172.21.15.33:0/1112489630 conn(0x7f179c06b3e0 msgr2=0x7f179c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 shutdown_connections 2024-04-25T01:08:44.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.569+0000 7f179bfff640 1 -- 172.21.15.33:0/1112489630 wait complete. 2024-04-25T01:08:44.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-04-25T01:08:44.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.581+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb580005860 con 0x7fb598034460 2024-04-25T01:08:44.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.581+0000 7fb595ffb640 1 -- 172.21.15.33:0/1667086309 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb59c0efd90 con 0x7fb598034460 2024-04-25T01:08:44.574 INFO:teuthology.orchestra.run.smithi033.stdout:64424509445 2024-04-25T01:08:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb5840678c0 msgr2=0x7fb584069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb5840678c0 0x7fb584069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb59804cc30 tx=0x7fb590009190 comp rx=0 tx=0).stop 2024-04-25T01:08:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 msgr2=0x7fb59804c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb59804c430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0ad040 tx=0x7fb59c019180 comp rx=0 tx=0).stop 2024-04-25T01:08:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 shutdown_connections 2024-04-25T01:08:44.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb5840678c0 0x7fb584069d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb59804f9b0 0x7fb5980e3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb598034840 0x7fb59804cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 --2- 172.21.15.33:0/1667086309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb598034460 0x7fb59804c430 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 >> 172.21.15.33:0/1667086309 conn(0x7fb598055360 msgr2=0x7fb5980e0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 shutdown_connections 2024-04-25T01:08:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:44.585+0000 7fb5a62c7640 1 -- 172.21.15.33:0/1667086309 wait complete. 2024-04-25T01:08:44.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509447 for osd.1 2024-04-25T01:08:44.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509448 for osd.4 2024-04-25T01:08:44.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509445 for osd.2 2024-04-25T01:08:45.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T01:08:45.544 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T01:08:45.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T01:08:45.553 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T01:08:45.571 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T01:08:45.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T01:08:45.579 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T01:08:45.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T01:08:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 Processor -- start 2024-04-25T01:08:45.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 -- start start 2024-04-25T01:08:45.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc308159da0 0x7fc30806d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080b2d90 0x7fc30806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3080aef00 0x7fc30815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc30815a180 con 0x7fc308159da0 2024-04-25T01:08:45.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc3080b3f30 con 0x7fc3080aef00 2024-04-25T01:08:45.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc30e5f6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3080aeab0 con 0x7fc3080b2d90 2024-04-25T01:08:45.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080b2d90 0x7fc30806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.721+0000 7fc3077fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080b2d90 0x7fc30806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60838/0 (socket says 172.21.15.33:60838) 2024-04-25T01:08:45.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc3077fe640 1 -- 172.21.15.33:0/2826269241 learned_addr learned my addr 172.21.15.33:0/2826269241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc307fff640 1 --2- 172.21.15.33:0/2826269241 >> v2:172.21.15.33:3300/0 conn(0x7fc308159da0 0x7fc30806d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc3077fe640 1 -- 172.21.15.33:0/2826269241 >> v2:172.21.15.138:3300/0 conn(0x7fc3080aef00 msgr2=0x7fc30815d8b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:08:45.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc3077fe640 1 --2- 172.21.15.33:0/2826269241 >> v2:172.21.15.138:3300/0 conn(0x7fc3080aef00 0x7fc30815d8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc3077fe640 1 -- 172.21.15.33:0/2826269241 >> v2:172.21.15.33:3300/0 conn(0x7fc308159da0 msgr2=0x7fc30806d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:45.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc3077fe640 1 --2- 172.21.15.33:0/2826269241 >> v2:172.21.15.33:3300/0 conn(0x7fc308159da0 0x7fc30806d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc3077fe640 1 -- 172.21.15.33:0/2826269241 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3080b41b0 con 0x7fc3080b2d90 2024-04-25T01:08:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.725+0000 7fc307fff640 1 --2- 172.21.15.33:0/2826269241 >> v2:172.21.15.33:3300/0 conn(0x7fc308159da0 0x7fc30806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc3077fe640 1 --2- 172.21.15.33:0/2826269241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080b2d90 0x7fc30806d710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc30806cb70 tx=0x7fc300020f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d0cf4e5a710ec59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc306ffd640 1 -- 172.21.15.33:0/2826269241 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3000af070 con 0x7fc3080b2d90 2024-04-25T01:08:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc306ffd640 1 -- 172.21.15.33:0/2826269241 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3000b3400 con 0x7fc3080b2d90 2024-04-25T01:08:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc306ffd640 1 -- 172.21.15.33:0/2826269241 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30000be60 con 0x7fc3080b2d90 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 -- 172.21.15.33:0/2826269241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080b2d90 msgr2=0x7fc30806d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 --2- 172.21.15.33:0/2826269241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080b2d90 0x7fc30806d710 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc30806cb70 tx=0x7fc300020f30 comp rx=0 tx=0).stop 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 -- 172.21.15.33:0/2826269241 shutdown_connections 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 --2- 172.21.15.33:0/2826269241 >> v2:172.21.15.138:3300/0 conn(0x7fc3080aef00 0x7fc30815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 --2- 172.21.15.33:0/2826269241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080b2d90 0x7fc30806d710 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 --2- 172.21.15.33:0/2826269241 >> v2:172.21.15.33:3300/0 conn(0x7fc308159da0 0x7fc30806d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 -- 172.21.15.33:0/2826269241 >> 172.21.15.33:0/2826269241 conn(0x7fc30806ba60 msgr2=0x7fc3080b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 -- 172.21.15.33:0/2826269241 shutdown_connections 2024-04-25T01:08:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.729+0000 7fc30e5f6640 1 -- 172.21.15.33:0/2826269241 wait complete. 2024-04-25T01:08:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 Processor -- start 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 -- start start 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080aef00 0x7fc3081569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3080b2d90 0x7fc3080ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308159da0 0x7fc3081481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080aef00 0x7fc3081569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080aef00 0x7fc3081569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:60842/0 (socket says 172.21.15.33:60842) 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 -- 172.21.15.33:0/930495496 learned_addr learned my addr 172.21.15.33:0/930495496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc3077fe640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3080b2d90 0x7fc3080ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc308070f30 con 0x7fc308159da0 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc308070db0 con 0x7fc3080b2d90 2024-04-25T01:08:45.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30e5f6640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3080710b0 con 0x7fc3080aef00 2024-04-25T01:08:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30cb93640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308159da0 0x7fc3081481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 -- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3080b2d90 msgr2=0x7fc3080ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3080b2d90 0x7fc3080ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 -- 172.21.15.33:0/930495496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308159da0 msgr2=0x7fc3081481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308159da0 0x7fc3081481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc308155970 con 0x7fc3080aef00 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc30cb93640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308159da0 0x7fc3081481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.733+0000 7fc307fff640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080aef00 0x7fc3081569d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e980 tx=0x7fc2f800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.737+0000 7fc304ff9640 1 -- 172.21.15.33:0/930495496 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8017070 con 0x7fc3080aef00 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.737+0000 7fc304ff9640 1 -- 172.21.15.33:0/930495496 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2f800ca00 con 0x7fc3080aef00 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.737+0000 7fc304ff9640 1 -- 172.21.15.33:0/930495496 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800cdb0 con 0x7fc3080aef00 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.737+0000 7fc30e5f6640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc308155c10 con 0x7fc3080aef00 2024-04-25T01:08:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.737+0000 7fc30e5f6640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081561d0 con 0x7fc3080aef00 2024-04-25T01:08:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.737+0000 7fc30e5f6640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc30804f580 con 0x7fc3080aef00 2024-04-25T01:08:45.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.737+0000 7fc304ff9640 1 -- 172.21.15.33:0/930495496 <== mon.2 v2:172.21.15.138:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc2f801a070 con 0x7fc3080aef00 2024-04-25T01:08:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afe37a640 1 Processor -- start 2024-04-25T01:08:45.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afe37a640 1 -- start start 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7fc304ff9640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2f0067ad0 0x7fc2f0069f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7fc304ff9640 1 -- 172.21.15.33:0/930495496 <== mon.2 v2:172.21.15.138:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc2f8085da0 con 0x7fc3080aef00 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7fc3077fe640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2f0067ad0 0x7fc2f0069f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afe37a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6af8143d90 0x7f6af80ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afe37a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6af80cc330 0x7f6af8071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afe37a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6af80ceef0 con 0x7f6af8071fb0 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afe37a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6af8072400 con 0x7f6af80cc330 2024-04-25T01:08:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afe37a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6af806ca90 con 0x7f6af8143d90 2024-04-25T01:08:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7fc3077fe640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2f0067ad0 0x7fc2f0069f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3000b4000 tx=0x7fc30000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afd378640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6af8143d90 0x7f6af80ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afdb79640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6af8071fb0 0x7f6af806d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33944/0 (socket says 172.21.15.33:33944) 2024-04-25T01:08:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afdb79640 1 -- 172.21.15.33:0/2997289154 learned_addr learned my addr 172.21.15.33:0/2997289154 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afcb77640 1 --2- 172.21.15.33:0/2997289154 >> v2:172.21.15.33:3300/0 conn(0x7f6af80cc330 0x7f6af8071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afcb77640 1 -- 172.21.15.33:0/2997289154 >> v2:172.21.15.138:3300/0 conn(0x7f6af8143d90 msgr2=0x7f6af80ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afcb77640 1 --2- 172.21.15.33:0/2997289154 >> v2:172.21.15.138:3300/0 conn(0x7f6af8143d90 0x7f6af80ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afcb77640 1 -- 172.21.15.33:0/2997289154 >> v1:172.21.15.33:6789/0 conn(0x7f6af8071fb0 legacy=0x7f6af806d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.741+0000 7f6afcb77640 1 -- 172.21.15.33:0/2997289154 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af806cd70 con 0x7f6af80cc330 2024-04-25T01:08:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afcb77640 1 --2- 172.21.15.33:0/2997289154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af80cc330 0x7f6af8071a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6af80b23f0 tx=0x7f6af401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d3c9c9f17e2b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6aeffff640 1 -- 172.21.15.33:0/2997289154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40b0070 con 0x7f6af80cc330 2024-04-25T01:08:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6aeffff640 1 -- 172.21.15.33:0/2997289154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af40ab600 con 0x7f6af80cc330 2024-04-25T01:08:45.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6aeffff640 1 -- 172.21.15.33:0/2997289154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af40ab9d0 con 0x7f6af80cc330 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 -- 172.21.15.33:0/2997289154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af80cc330 msgr2=0x7f6af8071a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 --2- 172.21.15.33:0/2997289154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af80cc330 0x7f6af8071a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6af80b23f0 tx=0x7f6af401c480 comp rx=0 tx=0).stop 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 -- 172.21.15.33:0/2997289154 shutdown_connections 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 --2- 172.21.15.33:0/2997289154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af80cc330 0x7f6af8071a70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 --2- 172.21.15.33:0/2997289154 >> v2:172.21.15.138:3300/0 conn(0x7f6af8143d90 0x7f6af80ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 -- 172.21.15.33:0/2997289154 >> 172.21.15.33:0/2997289154 conn(0x7f6af806ba40 msgr2=0x7f6af80b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 -- 172.21.15.33:0/2997289154 shutdown_connections 2024-04-25T01:08:45.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.745+0000 7f6afe37a640 1 -- 172.21.15.33:0/2997289154 wait complete. 2024-04-25T01:08:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 Processor -- start 2024-04-25T01:08:45.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 -- start start 2024-04-25T01:08:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7fe6f3fff640 1 Processor -- start 2024-04-25T01:08:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8071fb0 0x7f6af81557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80cc330 0x7f6af8127620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6af8143d90 0x7f6af8127b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6af806e8f0 con 0x7f6af8071fb0 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8071fb0 0x7f6af81557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8071fb0 0x7f6af81557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50170/0 (socket says 172.21.15.33:50170) 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 -- 172.21.15.33:0/1354227487 learned_addr learned my addr 172.21.15.33:0/1354227487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 -- 172.21.15.33:0/1354227487 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6af806e770 con 0x7f6af80cc330 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afe37a640 1 -- 172.21.15.33:0/1354227487 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6af806ea70 con 0x7f6af8143d90 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afcb77640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80cc330 0x7f6af8127620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afdb79640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6af8143d90 0x7f6af8127b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 -- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6af8143d90 msgr2=0x7f6af8127b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6af8143d90 0x7f6af8127b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 -- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80cc330 msgr2=0x7f6af8127620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80cc330 0x7f6af8127620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 -- 172.21.15.33:0/1354227487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af8155e90 con 0x7f6af8071fb0 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afcb77640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80cc330 0x7f6af8127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7f6afd378640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8071fb0 0x7f6af81557d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6af000efc0 tx=0x7f6af000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7fe6f3fff640 1 -- start start 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7fe6f3fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe6ec037510 0x7fe6ec0380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec049d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7fe6f3fff640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6ec007bd0 con 0x7fe6ec054db0 2024-04-25T01:08:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7fe6f3fff640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe6ec0e7c60 con 0x7fe6ec037510 2024-04-25T01:08:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.749+0000 7fe6f3fff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ec0522f0 con 0x7fe6ec054950 2024-04-25T01:08:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6cad54640 1 Processor -- start 2024-04-25T01:08:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f37fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe6ec054db0 0x7fe6ec052780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33960/0 (socket says 172.21.15.33:33960) 2024-04-25T01:08:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f37fe640 1 -- 172.21.15.33:0/2391487950 learned_addr learned my addr 172.21.15.33:0/2391487950 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f27fc640 1 --2- 172.21.15.33:0/2391487950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec049d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f2ffd640 1 --2- 172.21.15.33:0/2391487950 >> v2:172.21.15.138:3300/0 conn(0x7fe6ec037510 0x7fe6ec0380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7f6aedffb640 1 -- 172.21.15.33:0/1354227487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0018020 con 0x7f6af8071fb0 2024-04-25T01:08:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7f6afe37a640 1 -- 172.21.15.33:0/1354227487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af8156170 con 0x7f6af8071fb0 2024-04-25T01:08:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f1ffb640 1 -- 172.21.15.33:0/2391487950 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710336592 0 0) 0x7fe6ec007bd0 con 0x7fe6ec054db0 2024-04-25T01:08:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f1ffb640 1 -- 172.21.15.33:0/2391487950 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6dc003530 con 0x7fe6ec054db0 2024-04-25T01:08:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6cad54640 1 -- start start 2024-04-25T01:08:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7f6afe37a640 1 -- 172.21.15.33:0/1354227487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af8156730 con 0x7f6af8071fb0 2024-04-25T01:08:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7f6aedffb640 1 -- 172.21.15.33:0/1354227487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6af0014480 con 0x7f6af8071fb0 2024-04-25T01:08:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7f6aedffb640 1 -- 172.21.15.33:0/1354227487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6af0014850 con 0x7f6af8071fb0 2024-04-25T01:08:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7f6afe37a640 1 -- 172.21.15.33:0/1354227487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af804f580 con 0x7f6af8071fb0 2024-04-25T01:08:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fc304ff9640 1 -- 172.21.15.33:0/930495496 <== mon.2 v2:172.21.15.138:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc2f8055fc0 con 0x7fc3080aef00 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6cad54640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6bc054950 0x7fa6bc049d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6cad54640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc04a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6cad54640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6bc007bd0 con 0x7fa6bc037510 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6cad54640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa6bc0e7c60 con 0x7fa6bc054950 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6cad54640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6bc0522f0 con 0x7fa6bc054db0 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6c8af0640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa6bc037510 0x7fa6bc0380e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33968/0 (socket says 172.21.15.33:33968) 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6c8af0640 1 -- 172.21.15.33:0/2403916017 learned_addr learned my addr 172.21.15.33:0/2403916017 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6c92f1640 1 --2- 172.21.15.33:0/2403916017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc04a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6c92f1640 1 -- 172.21.15.33:0/2403916017 >> v2:172.21.15.33:3300/0 conn(0x7fa6bc054950 msgr2=0x7fa6bc049d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6c92f1640 1 --2- 172.21.15.33:0/2403916017 >> v2:172.21.15.33:3300/0 conn(0x7fa6bc054950 0x7fa6bc049d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6c92f1640 1 -- 172.21.15.33:0/2403916017 >> v1:172.21.15.33:6789/0 conn(0x7fa6bc037510 legacy=0x7fa6bc0380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fa6c92f1640 1 -- 172.21.15.33:0/2403916017 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc008430 con 0x7fa6bc054db0 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f27fc640 1 -- 172.21.15.33:0/2391487950 >> v2:172.21.15.138:3300/0 conn(0x7fe6ec037510 msgr2=0x7fe6ec0380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f27fc640 1 --2- 172.21.15.33:0/2391487950 >> v2:172.21.15.138:3300/0 conn(0x7fe6ec037510 0x7fe6ec0380e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f27fc640 1 -- 172.21.15.33:0/2391487950 >> v1:172.21.15.33:6789/0 conn(0x7fe6ec054db0 legacy=0x7fe6ec052780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f27fc640 1 -- 172.21.15.33:0/2391487950 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec008430 con 0x7fe6ec054950 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.753+0000 7fe6f2ffd640 1 --2- 172.21.15.33:0/2391487950 >> v2:172.21.15.138:3300/0 conn(0x7fe6ec037510 0x7fe6ec0380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fe6f27fc640 1 --2- 172.21.15.33:0/2391487950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec049d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80097c0 tx=0x7fe6e801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac1899f9d5ea2dfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fe6f1ffb640 1 -- 172.21.15.33:0/2391487950 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80b1070 con 0x7fe6ec054950 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fe6f1ffb640 1 -- 172.21.15.33:0/2391487950 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6e80ac630 con 0x7fe6ec054950 2024-04-25T01:08:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fe6f1ffb640 1 -- 172.21.15.33:0/2391487950 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80a8050 con 0x7fe6ec054950 2024-04-25T01:08:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7f6aedffb640 1 -- 172.21.15.33:0/1354227487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6af001b020 con 0x7f6af8071fb0 2024-04-25T01:08:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7f6aedffb640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6adc0678c0 0x7f6adc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7f6aedffb640 1 -- 172.21.15.33:0/1354227487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6af00872c0 con 0x7f6af8071fb0 2024-04-25T01:08:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7f6afcb77640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6adc0678c0 0x7f6adc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fe6f3fff640 1 -- 172.21.15.33:0/2391487950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 msgr2=0x7fe6ec049d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fe6f3fff640 1 --2- 172.21.15.33:0/2391487950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec049d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe6e80097c0 tx=0x7fe6e801c4c0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7f6afcb77640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6adc0678c0 0x7f6adc069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6af40ad000 tx=0x7f6af4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fa6c92f1640 1 --2- 172.21.15.33:0/2403916017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc04a290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc052470 tx=0x7fa6c4097a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=908f4eec9ed235b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fa6c37fe640 1 -- 172.21.15.33:0/2403916017 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c406e410 con 0x7fa6bc054db0 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fa6c37fe640 1 -- 172.21.15.33:0/2403916017 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6c406e5b0 con 0x7fa6bc054db0 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.757+0000 7fa6c37fe640 1 -- 172.21.15.33:0/2403916017 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c406ea20 con 0x7fa6bc054db0 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 -- 172.21.15.33:0/2403916017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 msgr2=0x7fa6bc04a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 --2- 172.21.15.33:0/2403916017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc04a290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc052470 tx=0x7fa6c4097a50 comp rx=0 tx=0).stop 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 -- 172.21.15.33:0/2403916017 shutdown_connections 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 --2- 172.21.15.33:0/2403916017 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc04a290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 --2- 172.21.15.33:0/2403916017 >> v2:172.21.15.33:3300/0 conn(0x7fa6bc054950 0x7fa6bc049d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 -- 172.21.15.33:0/2403916017 >> 172.21.15.33:0/2403916017 conn(0x7fa6bc020050 msgr2=0x7fa6bc056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fe6f3fff640 1 -- 172.21.15.33:0/2391487950 shutdown_connections 2024-04-25T01:08:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fe6f3fff640 1 --2- 172.21.15.33:0/2391487950 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec049d50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fe6f3fff640 1 --2- 172.21.15.33:0/2391487950 >> v2:172.21.15.138:3300/0 conn(0x7fe6ec037510 0x7fe6ec0380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fe6f3fff640 1 -- 172.21.15.33:0/2391487950 >> 172.21.15.33:0/2391487950 conn(0x7fe6ec020050 msgr2=0x7fe6ec056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 -- 172.21.15.33:0/2403916017 shutdown_connections 2024-04-25T01:08:45.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fe6f3fff640 1 -- 172.21.15.33:0/2391487950 shutdown_connections 2024-04-25T01:08:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 -- 172.21.15.33:0/2403916017 wait complete. 2024-04-25T01:08:45.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 Processor -- start 2024-04-25T01:08:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fe6f3fff640 1 -- 172.21.15.33:0/2391487950 wait complete. 2024-04-25T01:08:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 Processor -- start 2024-04-25T01:08:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 -- start start 2024-04-25T01:08:45.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f6aedffb640 1 -- 172.21.15.33:0/1354227487 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6af0022030 con 0x7f6af8071fb0 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.761+0000 7fa6cad54640 1 -- start start 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6bc037510 0x7fa6bc185e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6bc054950 0x7fa6bc056140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc189110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc0577a0 con 0x7fa6bc037510 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa6bc057620 con 0x7fa6bc054950 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa6bc052f70 con 0x7fa6bc054db0 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6bc037510 0x7fa6bc185e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6bc037510 0x7fa6bc185e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50184/0 (socket says 172.21.15.33:50184) 2024-04-25T01:08:45.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 -- 172.21.15.33:0/3195073646 learned_addr learned my addr 172.21.15.33:0/3195073646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c3fff640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6bc054950 0x7fa6bc056140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 Processor -- start 2024-04-25T01:08:45.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 -- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 msgr2=0x7fa6bc189110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc189110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 -- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6bc054950 msgr2=0x7fa6bc056140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6bc054950 0x7fa6bc056140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 -- 172.21.15.33:0/3195073646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6bc188950 con 0x7fa6bc037510 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6ec037510 0x7fe6ec050640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec0cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec054db0 0x7fe6ec0d8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec038620 con 0x7fe6ec054db0 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ec04a6a0 con 0x7fe6ec037510 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe6ec057620 con 0x7fe6ec054950 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec054db0 0x7fe6ec0d8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec054db0 0x7fe6ec0d8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50186/0 (socket says 172.21.15.33:50186) 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 -- 172.21.15.33:0/3165560274 learned_addr learned my addr 172.21.15.33:0/3165560274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f2ffd640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6ec037510 0x7fe6ec050640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f27fc640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec0cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 -- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 msgr2=0x7fe6ec0cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec0cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 -- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6ec037510 msgr2=0x7fe6ec050640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6ec037510 0x7fe6ec050640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 -- 172.21.15.33:0/3165560274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ec0d05f0 con 0x7fe6ec054db0 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f27fc640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec0cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f2ffd640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6ec037510 0x7fe6ec050640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f37fe640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec054db0 0x7fe6ec0d8a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6f406a000 tx=0x7fe6f40685e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c8af0640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6bc037510 0x7fa6bc185e30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000c500 tx=0x7fa6b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6db7fe640 1 -- 172.21.15.33:0/3165560274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4073880 con 0x7fe6ec054db0 2024-04-25T01:08:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c17fa640 1 -- 172.21.15.33:0/3195073646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0017070 con 0x7fa6bc037510 2024-04-25T01:08:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c17fa640 1 -- 172.21.15.33:0/3195073646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b0005cf0 con 0x7fa6bc037510 2024-04-25T01:08:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6bc04e2c0 con 0x7fa6bc037510 2024-04-25T01:08:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6bc188bf0 con 0x7fa6bc037510 2024-04-25T01:08:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6db7fe640 1 -- 172.21.15.33:0/3165560274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6f4073b40 con 0x7fe6ec054db0 2024-04-25T01:08:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6db7fe640 1 -- 172.21.15.33:0/3165560274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f4073f70 con 0x7fe6ec054db0 2024-04-25T01:08:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 -- 172.21.15.33:0/3165560274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ec0d0890 con 0x7fe6ec054db0 2024-04-25T01:08:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fe6f3fff640 1 -- 172.21.15.33:0/3165560274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ec04f080 con 0x7fe6ec054db0 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 -- start start 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c000b58b0 0x7f1c000b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c000b1ad0 0x7f1c000b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c0009c6b0 0x7f1c000d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1c000b71c0 con 0x7f1c000b1ad0 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1c000b68a0 con 0x7f1c000b58b0 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1c05886640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0009c220 con 0x7f1c0009c6b0 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7f1bfeffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c000b58b0 0x7f1c000b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfeffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1c000b58b0 0x7f1c000b6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52376/0 (socket says 172.21.15.33:52376) 2024-04-25T01:08:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfeffd640 1 -- 172.21.15.33:0/1872458432 learned_addr learned my addr 172.21.15.33:0/1872458432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.765+0000 7fa6c17fa640 1 -- 172.21.15.33:0/3195073646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00072f0 con 0x7fa6bc037510 2024-04-25T01:08:45.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfe7fc640 1 --2- 172.21.15.33:0/1872458432 >> v2:172.21.15.33:3300/0 conn(0x7f1c000b1ad0 0x7f1c000b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfeffd640 1 -- 172.21.15.33:0/1872458432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c0009c6b0 msgr2=0x7f1c000d1a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T01:08:45.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfeffd640 1 --2- 172.21.15.33:0/1872458432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c0009c6b0 0x7f1c000d1a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fe6f3fff640 1 -- 172.21.15.33:0/3165560274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ec0057f0 con 0x7fe6ec054db0 2024-04-25T01:08:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fe6db7fe640 1 -- 172.21.15.33:0/3165560274 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe6f4071040 con 0x7fe6ec054db0 2024-04-25T01:08:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfeffd640 1 -- 172.21.15.33:0/1872458432 >> v2:172.21.15.33:3300/0 conn(0x7f1c000b1ad0 msgr2=0x7f1c000b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:45.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfeffd640 1 --2- 172.21.15.33:0/1872458432 >> v2:172.21.15.33:3300/0 conn(0x7f1c000b1ad0 0x7f1c000b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfeffd640 1 -- 172.21.15.33:0/1872458432 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0009bef0 con 0x7f1c000b58b0 2024-04-25T01:08:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7f1bfe7fc640 1 --2- 172.21.15.33:0/1872458432 >> v2:172.21.15.33:3300/0 conn(0x7f1c000b1ad0 0x7f1c000b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fe6db7fe640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe6dc067de0 0x7fe6dc06a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fe6db7fe640 1 -- 172.21.15.33:0/3165560274 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe6f4101910 con 0x7fe6ec054db0 2024-04-25T01:08:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fa6c17fa640 1 -- 172.21.15.33:0/3195073646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6b001a070 con 0x7fa6bc037510 2024-04-25T01:08:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fe6f2ffd640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe6dc067de0 0x7fe6dc06a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fa6c17fa640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6a8067990 0x7fa6a8069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fa6c17fa640 1 -- 172.21.15.33:0/3195073646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa6b0085220 con 0x7fa6bc037510 2024-04-25T01:08:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fa6c3fff640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6a8067990 0x7fa6a8069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fe6f2ffd640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe6dc067de0 0x7fe6dc06a2a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400a9b0 tx=0x7fe6e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.769+0000 7fa6c3fff640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6a8067990 0x7fa6a8069e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc008ab0 tx=0x7fa6b8009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6bc188df0 con 0x7fa6bc037510 2024-04-25T01:08:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7f1bfeffd640 1 --2- 172.21.15.33:0/1872458432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c000b58b0 0x7f1c000b6c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c0008caa0 tx=0x7f1bf001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99832429fb6c5b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7f1bfdffb640 1 -- 172.21.15.33:0/1872458432 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf00aa680 con 0x7f1c000b58b0 2024-04-25T01:08:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7f1bfdffb640 1 -- 172.21.15.33:0/1872458432 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf00af070 con 0x7f1c000b58b0 2024-04-25T01:08:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7f1c05886640 1 -- 172.21.15.33:0/1872458432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c000b58b0 msgr2=0x7f1c000b6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7f1bfdffb640 1 -- 172.21.15.33:0/1872458432 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf00aa680 con 0x7f1c000b58b0 2024-04-25T01:08:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7f1c05886640 1 --2- 172.21.15.33:0/1872458432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c000b58b0 0x7f1c000b6c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c0008caa0 tx=0x7f1bf001c540 comp rx=0 tx=0).stop 2024-04-25T01:08:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.777+0000 7fe6db7fe640 1 -- 172.21.15.33:0/3165560274 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe6f40d2070 con 0x7fe6ec054db0 2024-04-25T01:08:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 -- 172.21.15.33:0/1872458432 shutdown_connections 2024-04-25T01:08:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 --2- 172.21.15.33:0/1872458432 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c0009c6b0 0x7f1c000d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 --2- 172.21.15.33:0/1872458432 >> v2:172.21.15.33:3300/0 conn(0x7f1c000b1ad0 0x7f1c000b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 --2- 172.21.15.33:0/1872458432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c000b58b0 0x7f1c000b6c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 -- 172.21.15.33:0/1872458432 >> 172.21.15.33:0/1872458432 conn(0x7f1c000d08b0 msgr2=0x7f1c000d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 -- 172.21.15.33:0/1872458432 shutdown_connections 2024-04-25T01:08:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 -- 172.21.15.33:0/1872458432 wait complete. 2024-04-25T01:08:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 Processor -- start 2024-04-25T01:08:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.785+0000 7f1c05886640 1 -- start start 2024-04-25T01:08:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7fa6c17fa640 1 -- 172.21.15.33:0/3195073646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa6bc188df0 con 0x7fa6bc037510 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c0009c6b0 0x7f1c00153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c000b1ad0 0x7f1c00154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c000b58b0 0x7f1c00157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0009cb20 con 0x7f1c000b1ad0 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c000b76d0 con 0x7f1c0009c6b0 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0009cca0 con 0x7f1c000b58b0 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c000b1ad0 0x7f1c00154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c000b1ad0 0x7f1c00154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50200/0 (socket says 172.21.15.33:50200) 2024-04-25T01:08:45.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 -- 172.21.15.33:0/2260358388 learned_addr learned my addr 172.21.15.33:0/2260358388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfeffd640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c0009c6b0 0x7f1c00153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 -- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c000b58b0 msgr2=0x7f1c00157ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:08:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c000b58b0 0x7f1c00157ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 -- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c0009c6b0 msgr2=0x7f1c00153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c0009c6b0 0x7f1c00153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 -- 172.21.15.33:0/2260358388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c000cdeb0 con 0x7f1c000b1ad0 2024-04-25T01:08:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1bfe7fc640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c000b1ad0 0x7f1c00154270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400ebe0 tx=0x7f1bf400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1be7fff640 1 -- 172.21.15.33:0/2260358388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4017070 con 0x7f1c000b1ad0 2024-04-25T01:08:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0009e9a0 con 0x7f1c000b1ad0 2024-04-25T01:08:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c000cb0a0 con 0x7f1c000b1ad0 2024-04-25T01:08:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1be7fff640 1 -- 172.21.15.33:0/2260358388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf400cc30 con 0x7f1c000b1ad0 2024-04-25T01:08:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.789+0000 7f1be7fff640 1 -- 172.21.15.33:0/2260358388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf4013470 con 0x7f1c000b1ad0 2024-04-25T01:08:45.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.793+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be00052a0 con 0x7f1c000b1ad0 2024-04-25T01:08:45.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.793+0000 7f1be7fff640 1 -- 172.21.15.33:0/2260358388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1bf401a070 con 0x7f1c000b1ad0 2024-04-25T01:08:45.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.793+0000 7f1be7fff640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1bec0678c0 0x7f1bec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.793+0000 7f1be7fff640 1 -- 172.21.15.33:0/2260358388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1bf4054070 con 0x7f1c000b1ad0 2024-04-25T01:08:45.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.793+0000 7f1bfeffd640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1bec0678c0 0x7f1bec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.793+0000 7f1bfeffd640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1bec0678c0 0x7f1bec069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00ae000 tx=0x7f1bf00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.801+0000 7f1be7fff640 1 -- 172.21.15.33:0/2260358388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1bf4055fd0 con 0x7f1c000b1ad0 2024-04-25T01:08:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.805+0000 7f34b4b2f640 1 Processor -- start 2024-04-25T01:08:45.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.805+0000 7f34b4b2f640 1 -- start start 2024-04-25T01:08:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.805+0000 7f34b4b2f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f34b0072790 0x7f34b00cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.805+0000 7f34b4b2f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b006f0c0 con 0x7f34b0162010 2024-04-25T01:08:45.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.805+0000 7f34b4b2f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34b006f3a0 con 0x7f34b006d200 2024-04-25T01:08:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.805+0000 7f34b4b2f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f34b0072000 con 0x7f34b0072790 2024-04-25T01:08:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34af7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f34b0162010 0x7f34b006caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33978/0 (socket says 172.21.15.33:33978) 2024-04-25T01:08:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34af7fe640 1 -- 172.21.15.33:0/805668123 learned_addr learned my addr 172.21.15.33:0/805668123 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34aeffd640 1 --2- 172.21.15.33:0/805668123 >> v2:172.21.15.33:3300/0 conn(0x7f34b0072790 0x7f34b00cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34aeffd640 1 -- 172.21.15.33:0/805668123 >> v1:172.21.15.138:6789/0 conn(0x7f34b006d200 legacy=0x7f34b00b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34aeffd640 1 -- 172.21.15.33:0/805668123 >> v1:172.21.15.33:6789/0 conn(0x7f34b0162010 legacy=0x7f34b006caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34aeffd640 1 -- 172.21.15.33:0/805668123 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b0071c90 con 0x7f34b0072790 2024-04-25T01:08:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34aeffd640 1 --2- 172.21.15.33:0/805668123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b0072790 0x7f34b00cf0e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f34a00097c0 tx=0x7f34a001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715c67e40de1706e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f34ae7fc640 1 -- 172.21.15.33:0/805668123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00b0070 con 0x7f34b0072790 2024-04-25T01:08:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f34ae7fc640 1 -- 172.21.15.33:0/805668123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a00ab600 con 0x7f34b0072790 2024-04-25T01:08:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f34ae7fc640 1 -- 172.21.15.33:0/805668123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00ab9d0 con 0x7f34b0072790 2024-04-25T01:08:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f4630a22640 1 Processor -- start 2024-04-25T01:08:45.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f4630a22640 1 -- start start 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f4630a22640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f462c15c6a0 0x7f462c071a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f4630a22640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c162e90 0x7f462c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f4630a22640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f462c166a60 con 0x7f462c071fb0 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f4630a22640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f462c163300 con 0x7f462c15c6a0 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.809+0000 7f4630a22640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f462c163520 con 0x7f462c162e90 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462ad9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f462c15c6a0 0x7f462c071a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462a59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f462c071fb0 0x7f462c162780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33992/0 (socket says 172.21.15.33:33992) 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462a59c640 1 -- 172.21.15.33:0/3774134735 learned_addr learned my addr 172.21.15.33:0/3774134735 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462b59e640 1 --2- 172.21.15.33:0/3774134735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c162e90 0x7f462c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462b59e640 1 -- 172.21.15.33:0/3774134735 >> v2:172.21.15.33:3300/0 conn(0x7f462c15c6a0 msgr2=0x7f462c071a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f34b4b2f640 1 -- 172.21.15.33:0/805668123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b0072790 msgr2=0x7f34b00cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/805668123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b0072790 0x7f34b00cf0e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f34a00097c0 tx=0x7f34a001c410 comp rx=0 tx=0).stop 2024-04-25T01:08:45.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462b59e640 1 --2- 172.21.15.33:0/3774134735 >> v2:172.21.15.33:3300/0 conn(0x7f462c15c6a0 0x7f462c071a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462b59e640 1 -- 172.21.15.33:0/3774134735 >> v1:172.21.15.33:6789/0 conn(0x7f462c071fb0 legacy=0x7f462c162780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462b59e640 1 -- 172.21.15.33:0/3774134735 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c165ca0 con 0x7f462c162e90 2024-04-25T01:08:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.813+0000 7f462ad9d640 1 --2- 172.21.15.33:0/3774134735 >> v2:172.21.15.33:3300/0 conn(0x7f462c15c6a0 0x7f462c071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 -- 172.21.15.33:0/805668123 shutdown_connections 2024-04-25T01:08:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/805668123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b0072790 0x7f34b00cf0e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 -- 172.21.15.33:0/805668123 >> 172.21.15.33:0/805668123 conn(0x7f34b00b14e0 msgr2=0x7f34b00b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 -- 172.21.15.33:0/805668123 shutdown_connections 2024-04-25T01:08:45.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 -- 172.21.15.33:0/805668123 wait complete. 2024-04-25T01:08:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f462b59e640 1 --2- 172.21.15.33:0/3774134735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c162e90 0x7f462c0b2850 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f461c0097c0 tx=0x7f461c01c2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ca5b280f23ef182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4629d9b640 1 -- 172.21.15.33:0/3774134735 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0b0070 con 0x7f462c162e90 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4629d9b640 1 -- 172.21.15.33:0/3774134735 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461c0ab600 con 0x7f462c162e90 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4629d9b640 1 -- 172.21.15.33:0/3774134735 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0ab9d0 con 0x7f462c162e90 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 -- 172.21.15.33:0/3774134735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c162e90 msgr2=0x7f462c0b2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 --2- 172.21.15.33:0/3774134735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c162e90 0x7f462c0b2850 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f461c0097c0 tx=0x7f461c01c2a0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 -- 172.21.15.33:0/3774134735 shutdown_connections 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 --2- 172.21.15.33:0/3774134735 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c162e90 0x7f462c0b2850 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 --2- 172.21.15.33:0/3774134735 >> v2:172.21.15.33:3300/0 conn(0x7f462c15c6a0 0x7f462c071a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 -- 172.21.15.33:0/3774134735 >> 172.21.15.33:0/3774134735 conn(0x7f462c06ba40 msgr2=0x7f462c0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 Processor -- start 2024-04-25T01:08:45.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 -- 172.21.15.33:0/3774134735 shutdown_connections 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 -- start start 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b006d200 0x7f34b01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f34b4b2f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b0072790 0x7f34b00ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34b4b2f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b0162010 0x7f34b00abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34b4b2f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00b3880 con 0x7f34b006d200 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34b4b2f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f34b00b3700 con 0x7f34b0072790 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34b4b2f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f34b00b3a00 con 0x7f34b0162010 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b0072790 0x7f34b00ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b0072790 0x7f34b00ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:52402/0 (socket says 172.21.15.33:52402) 2024-04-25T01:08:45.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 -- 172.21.15.33:0/2374321152 learned_addr learned my addr 172.21.15.33:0/2374321152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34affff640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b0162010 0x7f34b00abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.817+0000 7f4630a22640 1 -- 172.21.15.33:0/3774134735 wait complete. 2024-04-25T01:08:45.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 Processor -- start 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34af7fe640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b006d200 0x7f34b01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 -- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b0162010 msgr2=0x7f34b00abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b0162010 0x7f34b00abf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 -- 172.21.15.33:0/2374321152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b006d200 msgr2=0x7f34b01559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b006d200 0x7f34b01559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 -- 172.21.15.33:0/2374321152 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b00ab340 con 0x7f34b0072790 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34affff640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b0162010 0x7f34b00abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:08:45.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34aeffd640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b0072790 0x7f34b00ab9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f34a001a040 tx=0x7f34a00191b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 -- start start 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c071fb0 0x7f462c0aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c15c6a0 0x7f462c164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c162e90 0x7f462c14cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f462c15ca80 con 0x7f462c162e90 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f462c0b37f0 con 0x7f462c15c6a0 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f4630a22640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f462c16cf00 con 0x7f462c071fb0 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c162e90 0x7f462c14cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c162e90 0x7f462c14cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50210/0 (socket says 172.21.15.33:50210) 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 -- 172.21.15.33:0/2534729464 learned_addr learned my addr 172.21.15.33:0/2534729464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462ad9d640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c071fb0 0x7f462c0aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f349ffff640 1 -- 172.21.15.33:0/2374321152 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00b0040 con 0x7f34b0072790 2024-04-25T01:08:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b00ac530 con 0x7f34b0072790 2024-04-25T01:08:45.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b00acaf0 con 0x7f34b0072790 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462a59c640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c15c6a0 0x7f462c164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 -- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c071fb0 msgr2=0x7f462c0aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c071fb0 0x7f462c0aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 -- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c15c6a0 msgr2=0x7f462c164260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c15c6a0 0x7f462c164260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 -- 172.21.15.33:0/2534729464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c0aaa50 con 0x7f462c162e90 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462a59c640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c15c6a0 0x7f462c164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:45.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.821+0000 7f462b59e640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c162e90 0x7f462c14cee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f461c0ae040 tx=0x7f461c019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f349ffff640 1 -- 172.21.15.33:0/2374321152 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34a00ba070 con 0x7f34b0072790 2024-04-25T01:08:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f349ffff640 1 -- 172.21.15.33:0/2374321152 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a00b58c0 con 0x7f34b0072790 2024-04-25T01:08:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34940052a0 con 0x7f34b0072790 2024-04-25T01:08:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f46137fe640 1 -- 172.21.15.33:0/2534729464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0b8020 con 0x7f462c162e90 2024-04-25T01:08:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c15df10 con 0x7f462c162e90 2024-04-25T01:08:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c15e4d0 con 0x7f462c162e90 2024-04-25T01:08:45.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f46137fe640 1 -- 172.21.15.33:0/2534729464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461c0b4410 con 0x7f462c162e90 2024-04-25T01:08:45.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f46137fe640 1 -- 172.21.15.33:0/2534729464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0b0040 con 0x7f462c162e90 2024-04-25T01:08:45.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f349ffff640 1 -- 172.21.15.33:0/2374321152 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34a00b3070 con 0x7f34b0072790 2024-04-25T01:08:45.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c04f580 con 0x7f462c162e90 2024-04-25T01:08:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f349ffff640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3498067a50 0x7f3498069f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f349ffff640 1 -- 172.21.15.33:0/2374321152 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f34a011fea0 con 0x7f34b0072790 2024-04-25T01:08:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f34af7fe640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3498067a50 0x7f3498069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.829+0000 7f34af7fe640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3498067a50 0x7f3498069f10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000f70 tx=0x7f34a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f46137fe640 1 -- 172.21.15.33:0/2534729464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f461c0bb020 con 0x7f462c162e90 2024-04-25T01:08:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f46137fe640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4618067990 0x7f4618069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.825+0000 7f46137fe640 1 -- 172.21.15.33:0/2534729464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f461c11fab0 con 0x7f462c162e90 2024-04-25T01:08:45.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.829+0000 7f462ad9d640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4618067990 0x7f4618069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.829+0000 7f462ad9d640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4618067990 0x7f4618069e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f462c16d920 tx=0x7f462400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.837+0000 7f46137fe640 1 -- 172.21.15.33:0/2534729464 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f461c0c2030 con 0x7f462c162e90 2024-04-25T01:08:45.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.837+0000 7f349ffff640 1 -- 172.21.15.33:0/2374321152 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f34a0016090 con 0x7f34b0072790 2024-04-25T01:08:45.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.861+0000 7f748cb81640 1 Processor -- start 2024-04-25T01:08:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.861+0000 7f748cb81640 1 -- start start 2024-04-25T01:08:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f748cb81640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74880aeec0 0x7f74880711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f748cb81640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748806d890 con 0x7f74881577c0 2024-04-25T01:08:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f748cb81640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f748806db10 con 0x7f748806d400 2024-04-25T01:08:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f748cb81640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f74880b3110 con 0x7f74880aeec0 2024-04-25T01:08:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f74877fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f74881577c0 0x7f74880ae7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33994/0 (socket says 172.21.15.33:33994) 2024-04-25T01:08:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f7486ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f74880aeec0 0x7f74880711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f74877fe640 1 -- 172.21.15.33:0/1254123468 learned_addr learned my addr 172.21.15.33:0/1254123468 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f7486ffd640 1 -- 172.21.15.33:0/1254123468 >> v1:172.21.15.138:6789/0 conn(0x7f748806d400 legacy=0x7f74880b35a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T01:08:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f7486ffd640 1 -- 172.21.15.33:0/1254123468 >> v1:172.21.15.33:6789/0 conn(0x7f74881577c0 legacy=0x7f74880ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.865+0000 7f7486ffd640 1 -- 172.21.15.33:0/1254123468 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74880b27e0 con 0x7f74880aeec0 2024-04-25T01:08:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f7486ffd640 1 --2- 172.21.15.33:0/1254123468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74880aeec0 0x7f74880711d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f74800064a0 tx=0x7f7480024f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df06f11a0565b66c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f74867fc640 1 -- 172.21.15.33:0/1254123468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74800b9070 con 0x7f74880aeec0 2024-04-25T01:08:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f74867fc640 1 -- 172.21.15.33:0/1254123468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74800b4600 con 0x7f74880aeec0 2024-04-25T01:08:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f748cb81640 1 -- 172.21.15.33:0/1254123468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74880aeec0 msgr2=0x7f74880711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f748cb81640 1 --2- 172.21.15.33:0/1254123468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74880aeec0 0x7f74880711d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f74800064a0 tx=0x7f7480024f70 comp rx=0 tx=0).stop 2024-04-25T01:08:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f748cb81640 1 -- 172.21.15.33:0/1254123468 shutdown_connections 2024-04-25T01:08:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f748cb81640 1 --2- 172.21.15.33:0/1254123468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74880aeec0 0x7f74880711d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f748cb81640 1 -- 172.21.15.33:0/1254123468 >> 172.21.15.33:0/1254123468 conn(0x7f74880b14e0 msgr2=0x7f74880b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:45.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f748cb81640 1 -- 172.21.15.33:0/1254123468 shutdown_connections 2024-04-25T01:08:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.869+0000 7f748cb81640 1 -- 172.21.15.33:0/1254123468 wait complete. 2024-04-25T01:08:45.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 Processor -- start 2024-04-25T01:08:45.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 -- start start 2024-04-25T01:08:45.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f748806d400 0x7f7488202ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74881577c0 0x7f7488072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7488072890 0x7f74880a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74881679a0 con 0x7f74881577c0 2024-04-25T01:08:45.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7488157ba0 con 0x7f7488072890 2024-04-25T01:08:45.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f748cb81640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7488167b20 con 0x7f748806d400 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f74877fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f748806d400 0x7f7488202ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74881577c0 0x7f7488072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74881577c0 0x7f7488072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50240/0 (socket says 172.21.15.33:50240) 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 -- 172.21.15.33:0/1238798517 learned_addr learned my addr 172.21.15.33:0/1238798517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 -- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f748806d400 msgr2=0x7f7488202ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f748806d400 0x7f7488202ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 -- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7488072890 msgr2=0x7f74880a9a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7488072890 0x7f74880a9a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 -- 172.21.15.33:0/1238798517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74880aa6b0 con 0x7f74881577c0 2024-04-25T01:08:45.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.873+0000 7f7486ffd640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74881577c0 0x7f7488072350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7480021b80 tx=0x7f7480021d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f746ffff640 1 -- 172.21.15.33:0/1238798517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74800b9030 con 0x7f74881577c0 2024-04-25T01:08:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74880aa990 con 0x7f74881577c0 2024-04-25T01:08:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74880aaf50 con 0x7f74881577c0 2024-04-25T01:08:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f746ffff640 1 -- 172.21.15.33:0/1238798517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f748001a520 con 0x7f74881577c0 2024-04-25T01:08:45.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f746ffff640 1 -- 172.21.15.33:0/1238798517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7480018070 con 0x7f74881577c0 2024-04-25T01:08:45.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f746ffff640 1 -- 172.21.15.33:0/1238798517 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f74800bc030 con 0x7f74881577c0 2024-04-25T01:08:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f746ffff640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7474067990 0x7f7474069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f746ffff640 1 -- 172.21.15.33:0/1238798517 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f74801205c0 con 0x7f74881577c0 2024-04-25T01:08:45.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.877+0000 7f74877fe640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7474067990 0x7f7474069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:45.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.881+0000 7f74877fe640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7474067990 0x7f7474069e50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7478000c10 tx=0x7f7478009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:45.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.885+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74680052a0 con 0x7f74881577c0 2024-04-25T01:08:45.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:45.893+0000 7f746ffff640 1 -- 172.21.15.33:0/1238798517 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f74800f5e20 con 0x7f74881577c0 2024-04-25T01:08:46.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.081+0000 7fc30e5f6640 1 -- 172.21.15.33:0/930495496 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc308156fc0 con 0x7fc3080aef00 2024-04-25T01:08:46.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.081+0000 7fc304ff9640 1 -- 172.21.15.33:0/930495496 <== mon.2 v2:172.21.15.138:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc2f805a2c0 con 0x7fc3080aef00 2024-04-25T01:08:46.076 INFO:teuthology.orchestra.run.smithi033.stdout:64424509449 2024-04-25T01:08:46.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 -- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2f0067ad0 msgr2=0x7fc2f0069f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2f0067ad0 0x7fc2f0069f90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc3000b4000 tx=0x7fc30000d040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 -- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080aef00 msgr2=0x7fc3081569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080aef00 0x7fc3081569d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800e980 tx=0x7fc2f800ee40 comp rx=0 tx=0).stop 2024-04-25T01:08:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 -- 172.21.15.33:0/930495496 shutdown_connections 2024-04-25T01:08:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2f0067ad0 0x7fc2f0069f90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308159da0 0x7fc3081481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3080b2d90 0x7fc3080ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 --2- 172.21.15.33:0/930495496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3080aef00 0x7fc3081569d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 -- 172.21.15.33:0/930495496 >> 172.21.15.33:0/930495496 conn(0x7fc30806ba60 msgr2=0x7fc3080b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 -- 172.21.15.33:0/930495496 shutdown_connections 2024-04-25T01:08:46.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.085+0000 7fc2ee7fc640 1 -- 172.21.15.33:0/930495496 wait complete. 2024-04-25T01:08:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.089+0000 7fe6f3fff640 1 -- 172.21.15.33:0/3165560274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe6ec054250 con 0x7fe6ec054db0 2024-04-25T01:08:46.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.089+0000 7fe6db7fe640 1 -- 172.21.15.33:0/3165560274 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6f40d1970 con 0x7fe6ec054db0 2024-04-25T01:08:46.085 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 -- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe6dc067de0 msgr2=0x7fe6dc06a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe6dc067de0 0x7fe6dc06a2a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400a9b0 tx=0x7fe6e4008040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 -- 172.21.15.33:0/3165560274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec054db0 msgr2=0x7fe6ec0d8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec054db0 0x7fe6ec0d8a50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe6f406a000 tx=0x7fe6f40685e0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 -- 172.21.15.33:0/3165560274 shutdown_connections 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe6dc067de0 0x7fe6dc06a2a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6ec054db0 0x7fe6ec0d8a50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe6ec054950 0x7fe6ec0cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 --2- 172.21.15.33:0/3165560274 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6ec037510 0x7fe6ec050640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 -- 172.21.15.33:0/3165560274 >> 172.21.15.33:0/3165560274 conn(0x7fe6ec020050 msgr2=0x7fe6ec04a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 -- 172.21.15.33:0/3165560274 shutdown_connections 2024-04-25T01:08:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.097+0000 7fe6d97fa640 1 -- 172.21.15.33:0/3165560274 wait complete. 2024-04-25T01:08:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.101+0000 7f6afe37a640 1 -- 172.21.15.33:0/1354227487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6af80b3860 con 0x7f6af8071fb0 2024-04-25T01:08:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.101+0000 7f6aedffb640 1 -- 172.21.15.33:0/1354227487 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6af0010040 con 0x7f6af8071fb0 2024-04-25T01:08:46.096 INFO:teuthology.orchestra.run.smithi033.stdout:55834574856 2024-04-25T01:08:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 -- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6adc0678c0 msgr2=0x7f6adc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6adc0678c0 0x7f6adc069d80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6af40ad000 tx=0x7f6af4008040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 -- 172.21.15.33:0/1354227487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8071fb0 msgr2=0x7f6af81557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8071fb0 0x7f6af81557d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6af000efc0 tx=0x7f6af000c490 comp rx=0 tx=0).stop 2024-04-25T01:08:46.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-04-25T01:08:46.104 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 -- 172.21.15.33:0/1354227487 shutdown_connections 2024-04-25T01:08:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6adc0678c0 0x7f6adc069d80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6af8143d90 0x7f6af8127b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6af80cc330 0x7f6af8127620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 --2- 172.21.15.33:0/1354227487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6af8071fb0 0x7f6af81557d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 -- 172.21.15.33:0/1354227487 >> 172.21.15.33:0/1354227487 conn(0x7f6af806ba40 msgr2=0x7f6af80b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 -- 172.21.15.33:0/1354227487 shutdown_connections 2024-04-25T01:08:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.109+0000 7f6ae37fe640 1 -- 172.21.15.33:0/1354227487 wait complete. 2024-04-25T01:08:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.113+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa6bc0559f0 con 0x7fa6bc037510 2024-04-25T01:08:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.113+0000 7fa6c17fa640 1 -- 172.21.15.33:0/3195073646 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa6b0055440 con 0x7fa6bc037510 2024-04-25T01:08:46.107 INFO:teuthology.orchestra.run.smithi033.stdout:64424509449 2024-04-25T01:08:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6a8067990 msgr2=0x7fa6a8069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6a8067990 0x7fa6a8069e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc008ab0 tx=0x7fa6b8009110 comp rx=0 tx=0).stop 2024-04-25T01:08:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6bc037510 msgr2=0x7fa6bc185e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6bc037510 0x7fa6bc185e30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000c500 tx=0x7fa6b000c9c0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 shutdown_connections 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6a8067990 0x7fa6a8069e50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6bc054db0 0x7fa6bc189110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6bc054950 0x7fa6bc056140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 --2- 172.21.15.33:0/3195073646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6bc037510 0x7fa6bc185e30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 >> 172.21.15.33:0/3195073646 conn(0x7fa6bc020050 msgr2=0x7fa6bc0354c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.117+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 shutdown_connections 2024-04-25T01:08:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.121+0000 7fa6cad54640 1 -- 172.21.15.33:0/3195073646 wait complete. 2024-04-25T01:08:46.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2024-04-25T01:08:46.117 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.137+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1be0005590 con 0x7f1c000b1ad0 2024-04-25T01:08:46.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-25T01:08:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.137+0000 7f1be7fff640 1 -- 172.21.15.33:0/2260358388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1bf405a2d0 con 0x7f1c000b1ad0 2024-04-25T01:08:46.130 INFO:teuthology.orchestra.run.smithi033.stdout:64424509448 2024-04-25T01:08:46.130 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1bec0678c0 msgr2=0x7f1bec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1bec0678c0 0x7f1bec069d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1bf00ae000 tx=0x7f1bf00a7000 comp rx=0 tx=0).stop 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c000b1ad0 msgr2=0x7f1c00154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c000b1ad0 0x7f1c00154270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400ebe0 tx=0x7f1bf400c520 comp rx=0 tx=0).stop 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 shutdown_connections 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1bec0678c0 0x7f1bec069d80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c000b58b0 0x7f1c00157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c000b1ad0 0x7f1c00154270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 --2- 172.21.15.33:0/2260358388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c0009c6b0 0x7f1c00153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 >> 172.21.15.33:0/2260358388 conn(0x7f1c000d08b0 msgr2=0x7f1c000d3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 shutdown_connections 2024-04-25T01:08:46.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.141+0000 7f1c05886640 1 -- 172.21.15.33:0/2260358388 wait complete. 2024-04-25T01:08:46.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.6 2024-04-25T01:08:46.135 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.145+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f462c0b3c70 con 0x7f462c162e90 2024-04-25T01:08:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f46137fe640 1 -- 172.21.15.33:0/2534729464 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f461c015fb0 con 0x7f462c162e90 2024-04-25T01:08:46.140 INFO:teuthology.orchestra.run.smithi033.stdout:64424509450 2024-04-25T01:08:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4618067990 msgr2=0x7f4618069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4618067990 0x7f4618069e50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f462c16d920 tx=0x7f462400b040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c162e90 msgr2=0x7f462c14cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c162e90 0x7f462c14cee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f461c0ae040 tx=0x7f461c019040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 shutdown_connections 2024-04-25T01:08:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4618067990 0x7f4618069e50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c162e90 0x7f462c14cee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f3494005860 con 0x7f34b0072790 2024-04-25T01:08:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c15c6a0 0x7f462c164260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 --2- 172.21.15.33:0/2534729464 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f462c071fb0 0x7f462c0aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 >> 172.21.15.33:0/2534729464 conn(0x7f462c06ba40 msgr2=0x7f462c0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 shutdown_connections 2024-04-25T01:08:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f4630a22640 1 -- 172.21.15.33:0/2534729464 wait complete. 2024-04-25T01:08:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.149+0000 7f349ffff640 1 -- 172.21.15.33:0/2374321152 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f34a00f0930 con 0x7f34b0072790 2024-04-25T01:08:46.144 INFO:teuthology.orchestra.run.smithi033.stdout:64424509449 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3498067a50 msgr2=0x7f3498069f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3498067a50 0x7f3498069f10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34a4000f70 tx=0x7f34a4009040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b0072790 msgr2=0x7f34b00ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b0072790 0x7f34b00ab9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f34a001a040 tx=0x7f34a00191b0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 shutdown_connections 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3498067a50 0x7f3498069f10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f34b0162010 0x7f34b00abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f34b0072790 0x7f34b00ab9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 --2- 172.21.15.33:0/2374321152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34b006d200 0x7f34b01559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 >> 172.21.15.33:0/2374321152 conn(0x7f34b00b14e0 msgr2=0x7f34b00b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 shutdown_connections 2024-04-25T01:08:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.153+0000 7f34b4b2f640 1 -- 172.21.15.33:0/2374321152 wait complete. 2024-04-25T01:08:46.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.7 2024-04-25T01:08:46.151 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7468005590 con 0x7f74881577c0 2024-04-25T01:08:46.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f746ffff640 1 -- 172.21.15.33:0/1238798517 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f748001ecf0 con 0x7f74881577c0 2024-04-25T01:08:46.157 INFO:teuthology.orchestra.run.smithi033.stdout:64424509447 2024-04-25T01:08:46.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7474067990 msgr2=0x7f7474069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7474067990 0x7f7474069e50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7478000c10 tx=0x7f7478009040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74881577c0 msgr2=0x7f7488072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74881577c0 0x7f7488072350 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7480021b80 tx=0x7f7480021d10 comp rx=0 tx=0).stop 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 shutdown_connections 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7474067990 0x7f7474069e50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7488072890 0x7f74880a9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74881577c0 0x7f7488072350 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 --2- 172.21.15.33:0/1238798517 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f748806d400 0x7f7488202ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 >> 172.21.15.33:0/1238798517 conn(0x7f74880b14e0 msgr2=0x7f748815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 shutdown_connections 2024-04-25T01:08:46.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.165+0000 7f748cb81640 1 -- 172.21.15.33:0/1238798517 wait complete. 2024-04-25T01:08:46.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.4 2024-04-25T01:08:46.168 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.1 2024-04-25T01:08:46.170 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.176 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-04-25T01:08:46.176 DEBUG:teuthology.parallel:result is None 2024-04-25T01:08:46.176 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T01:08:46.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:08:46.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 Processor -- start 2024-04-25T01:08:46.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 -- start start 2024-04-25T01:08:46.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c159b00 0x7f875c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f875c0b5e90 0x7f875c15d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f875c0cdc20 0x7f875c15db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f875c159ee0 con 0x7f875c0b5e90 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f875c06c8b0 con 0x7f875c0cdc20 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f8761f84640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f875c0b6330 con 0x7f875c159b00 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f875c0b5e90 0x7f875c15d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f875c0b5e90 0x7f875c15d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50244/0 (socket says 172.21.15.33:50244) 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 -- 172.21.15.33:0/4272285962 learned_addr learned my addr 172.21.15.33:0/4272285962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875bfff640 1 --2- 172.21.15.33:0/4272285962 >> v2:172.21.15.138:3300/0 conn(0x7f875c0cdc20 0x7f875c15db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875b7fe640 1 --2- 172.21.15.33:0/4272285962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c159b00 0x7f875c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 -- 172.21.15.33:0/4272285962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c159b00 msgr2=0x7f875c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 --2- 172.21.15.33:0/4272285962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c159b00 0x7f875c0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 -- 172.21.15.33:0/4272285962 >> v2:172.21.15.138:3300/0 conn(0x7f875c0cdc20 msgr2=0x7f875c15db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:46.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 --2- 172.21.15.33:0/4272285962 >> v2:172.21.15.138:3300/0 conn(0x7f875c0cdc20 0x7f875c15db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.333+0000 7f875affd640 1 -- 172.21.15.33:0/4272285962 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c073400 con 0x7f875c0b5e90 2024-04-25T01:08:46.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f875affd640 1 --2- 172.21.15.33:0/4272285962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c0b5e90 0x7f875c15d600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f875c0ce000 tx=0x7f875001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6224b0a725023230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:46.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f875a7fc640 1 -- 172.21.15.33:0/4272285962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500aa5f0 con 0x7f875c0b5e90 2024-04-25T01:08:46.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f875a7fc640 1 -- 172.21.15.33:0/4272285962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87500af070 con 0x7f875c0b5e90 2024-04-25T01:08:46.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f875a7fc640 1 -- 172.21.15.33:0/4272285962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87500aa9b0 con 0x7f875c0b5e90 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- 172.21.15.33:0/4272285962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c0b5e90 msgr2=0x7f875c15d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 --2- 172.21.15.33:0/4272285962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c0b5e90 0x7f875c15d600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f875c0ce000 tx=0x7f875001c4f0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- 172.21.15.33:0/4272285962 shutdown_connections 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 --2- 172.21.15.33:0/4272285962 >> v2:172.21.15.138:3300/0 conn(0x7f875c0cdc20 0x7f875c15db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 --2- 172.21.15.33:0/4272285962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c0b5e90 0x7f875c15d600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 --2- 172.21.15.33:0/4272285962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c159b00 0x7f875c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- 172.21.15.33:0/4272285962 >> 172.21.15.33:0/4272285962 conn(0x7f875c0b50c0 msgr2=0x7f875c0b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- 172.21.15.33:0/4272285962 shutdown_connections 2024-04-25T01:08:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- 172.21.15.33:0/4272285962 wait complete. 2024-04-25T01:08:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 Processor -- start 2024-04-25T01:08:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- start start 2024-04-25T01:08:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c0b5e90 0x7f875c1455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c0cdc20 0x7f875c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c159b00 0x7f875c0b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f875c15ce80 con 0x7f875c159b00 2024-04-25T01:08:46.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f875c15cd00 con 0x7f875c0b5e90 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f8761f84640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f875c15d000 con 0x7f875c0cdc20 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f875bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c159b00 0x7f875c0b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f875bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c159b00 0x7f875c0b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50250/0 (socket says 172.21.15.33:50250) 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.337+0000 7f875bfff640 1 -- 172.21.15.33:0/1678226068 learned_addr learned my addr 172.21.15.33:0/1678226068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875affd640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c0cdc20 0x7f875c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875bfff640 1 -- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c0cdc20 msgr2=0x7f875c0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875bfff640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c0cdc20 0x7f875c0b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875bfff640 1 -- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c0b5e90 msgr2=0x7f875c1455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875bfff640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c0b5e90 0x7f875c1455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875bfff640 1 -- 172.21.15.33:0/1678226068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875c152260 con 0x7f875c159b00 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875bfff640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c159b00 0x7f875c0b0d60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f874c007d60 tx=0x7f874c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8743fff640 1 -- 172.21.15.33:0/1678226068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c014420 con 0x7f875c159b00 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8743fff640 1 -- 172.21.15.33:0/1678226068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f874c019070 con 0x7f875c159b00 2024-04-25T01:08:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8743fff640 1 -- 172.21.15.33:0/1678226068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874c014820 con 0x7f875c159b00 2024-04-25T01:08:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875c152500 con 0x7f875c159b00 2024-04-25T01:08:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f875c145ae0 con 0x7f875c159b00 2024-04-25T01:08:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f875c069040 con 0x7f875c159b00 2024-04-25T01:08:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8743fff640 1 -- 172.21.15.33:0/1678226068 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f874c0149c0 con 0x7f875c159b00 2024-04-25T01:08:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8743fff640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87480678c0 0x7f8748069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f8743fff640 1 -- 172.21.15.33:0/1678226068 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f874c086750 con 0x7f875c159b00 2024-04-25T01:08:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875b7fe640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87480678c0 0x7f8748069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.341+0000 7f875b7fe640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87480678c0 0x7f8748069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f875400b850 tx=0x7f8754008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:46.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.345+0000 7f8743fff640 1 -- 172.21.15.33:0/1678226068 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f874c056fe0 con 0x7f875c159b00 2024-04-25T01:08:46.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.497+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f875c15af40 con 0x7f87480678c0 2024-04-25T01:08:46.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.521+0000 7f8743fff640 1 -- 172.21.15.33:0/1678226068 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f875c15af40 con 0x7f87480678c0 2024-04-25T01:08:46.518 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:46.520 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:08:46.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87480678c0 msgr2=0x7f8748069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87480678c0 0x7f8748069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f875400b850 tx=0x7f8754008040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c159b00 msgr2=0x7f875c0b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c159b00 0x7f875c0b0d60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f874c007d60 tx=0x7f874c00e4b0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 shutdown_connections 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87480678c0 0x7f8748069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875c159b00 0x7f875c0b0d60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f875c0cdc20 0x7f875c0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 --2- 172.21.15.33:0/1678226068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875c0b5e90 0x7f875c1455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 >> 172.21.15.33:0/1678226068 conn(0x7f875c0b50c0 msgr2=0x7f875c0b5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 shutdown_connections 2024-04-25T01:08:46.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.525+0000 7f8761f84640 1 -- 172.21.15.33:0/1678226068 wait complete. 2024-04-25T01:08:46.537 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T01:08:45.092884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223588,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751132,"statfs":{"total":773094113280,"available":772865159168,"internally_reserved":0,"allocated":3317760,"data_stored":2009720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005168"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830172+0000","last_change":"2024-04-25T01:08:23.464284+0000","last_active":"2024-04-25T01:08:29.830172+0000","last_peered":"2024-04-25T01:08:29.830172+0000","last_clean":"2024-04-25T01:08:29.830172+0000","last_became_active":"2024-04-25T01:08:23.464018+0000","last_became_peered":"2024-04-25T01:08:23.464018+0000","last_unstale":"2024-04-25T01:08:29.830172+0000","last_undegraded":"2024-04-25T01:08:29.830172+0000","last_fullsized":"2024-04-25T01:08:29.830172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:27.195443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509770+0000","last_change":"2024-04-25T01:08:24.479420+0000","last_active":"2024-04-25T01:08:29.509770+0000","last_peered":"2024-04-25T01:08:29.509770+0000","last_clean":"2024-04-25T01:08:29.509770+0000","last_became_active":"2024-04-25T01:08:24.479003+0000","last_became_peered":"2024-04-25T01:08:24.479003+0000","last_unstale":"2024-04-25T01:08:29.509770+0000","last_undegraded":"2024-04-25T01:08:29.509770+0000","last_fullsized":"2024-04-25T01:08:29.509770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:10.309810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507734+0000","last_change":"2024-04-25T01:08:28.865743+0000","last_active":"2024-04-25T01:08:29.507734+0000","last_peered":"2024-04-25T01:08:29.507734+0000","last_clean":"2024-04-25T01:08:29.507734+0000","last_became_active":"2024-04-25T01:08:28.864434+0000","last_became_peered":"2024-04-25T01:08:28.864434+0000","last_unstale":"2024-04-25T01:08:29.507734+0000","last_undegraded":"2024-04-25T01:08:29.507734+0000","last_fullsized":"2024-04-25T01:08:29.507734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.379931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510114+0000","last_change":"2024-04-25T01:08:23.447876+0000","last_active":"2024-04-25T01:08:29.510114+0000","last_peered":"2024-04-25T01:08:29.510114+0000","last_clean":"2024-04-25T01:08:29.510114+0000","last_became_active":"2024-04-25T01:08:23.447588+0000","last_became_peered":"2024-04-25T01:08:23.447588+0000","last_unstale":"2024-04-25T01:08:29.510114+0000","last_undegraded":"2024-04-25T01:08:29.510114+0000","last_fullsized":"2024-04-25T01:08:29.510114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830021+0000","last_change":"2024-04-25T01:08:24.494422+0000","last_active":"2024-04-25T01:08:29.830021+0000","last_peered":"2024-04-25T01:08:29.830021+0000","last_clean":"2024-04-25T01:08:29.830021+0000","last_became_active":"2024-04-25T01:08:24.494035+0000","last_became_peered":"2024-04-25T01:08:24.494035+0000","last_unstale":"2024-04-25T01:08:29.830021+0000","last_undegraded":"2024-04-25T01:08:29.830021+0000","last_fullsized":"2024-04-25T01:08:29.830021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510126+0000","last_change":"2024-04-25T01:08:28.548777+0000","last_active":"2024-04-25T01:08:29.510126+0000","last_peered":"2024-04-25T01:08:29.510126+0000","last_clean":"2024-04-25T01:08:29.510126+0000","last_became_active":"2024-04-25T01:08:28.545213+0000","last_became_peered":"2024-04-25T01:08:28.545213+0000","last_unstale":"2024-04-25T01:08:29.510126+0000","last_undegraded":"2024-04-25T01:08:29.510126+0000","last_fullsized":"2024-04-25T01:08:29.510126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:14.372448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826296+0000","last_change":"2024-04-25T01:08:23.483744+0000","last_active":"2024-04-25T01:08:29.826296+0000","last_peered":"2024-04-25T01:08:29.826296+0000","last_clean":"2024-04-25T01:08:29.826296+0000","last_became_active":"2024-04-25T01:08:23.483245+0000","last_became_peered":"2024-04-25T01:08:23.483245+0000","last_unstale":"2024-04-25T01:08:29.826296+0000","last_undegraded":"2024-04-25T01:08:29.826296+0000","last_fullsized":"2024-04-25T01:08:29.826296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:38.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826404+0000","last_change":"2024-04-25T01:08:24.480206+0000","last_active":"2024-04-25T01:08:29.826404+0000","last_peered":"2024-04-25T01:08:29.826404+0000","last_clean":"2024-04-25T01:08:29.826404+0000","last_became_active":"2024-04-25T01:08:24.479327+0000","last_became_peered":"2024-04-25T01:08:24.479327+0000","last_unstale":"2024-04-25T01:08:29.826404+0000","last_undegraded":"2024-04-25T01:08:29.826404+0000","last_fullsized":"2024-04-25T01:08:29.826404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.885275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510631+0000","last_change":"2024-04-25T01:08:28.545701+0000","last_active":"2024-04-25T01:08:29.510631+0000","last_peered":"2024-04-25T01:08:29.510631+0000","last_clean":"2024-04-25T01:08:29.510631+0000","last_became_active":"2024-04-25T01:08:28.544731+0000","last_became_peered":"2024-04-25T01:08:28.544731+0000","last_unstale":"2024-04-25T01:08:29.510631+0000","last_undegraded":"2024-04-25T01:08:29.510631+0000","last_fullsized":"2024-04-25T01:08:29.510631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.483124+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.482403+0000","last_became_peered":"2024-04-25T01:08:23.482403+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:22.095395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827570+0000","last_change":"2024-04-25T01:08:24.488697+0000","last_active":"2024-04-25T01:08:29.827570+0000","last_peered":"2024-04-25T01:08:29.827570+0000","last_clean":"2024-04-25T01:08:29.827570+0000","last_became_active":"2024-04-25T01:08:24.487850+0000","last_became_peered":"2024-04-25T01:08:24.487850+0000","last_unstale":"2024-04-25T01:08:29.827570+0000","last_undegraded":"2024-04-25T01:08:29.827570+0000","last_fullsized":"2024-04-25T01:08:29.827570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:57.185281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510492+0000","last_change":"2024-04-25T01:08:28.545916+0000","last_active":"2024-04-25T01:08:29.510492+0000","last_peered":"2024-04-25T01:08:29.510492+0000","last_clean":"2024-04-25T01:08:29.510492+0000","last_became_active":"2024-04-25T01:08:28.544666+0000","last_became_peered":"2024-04-25T01:08:28.544666+0000","last_unstale":"2024-04-25T01:08:29.510492+0000","last_undegraded":"2024-04-25T01:08:29.510492+0000","last_fullsized":"2024-04-25T01:08:29.510492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:03.657733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827605+0000","last_change":"2024-04-25T01:08:24.498408+0000","last_active":"2024-04-25T01:08:29.827605+0000","last_peered":"2024-04-25T01:08:29.827605+0000","last_clean":"2024-04-25T01:08:29.827605+0000","last_became_active":"2024-04-25T01:08:24.497888+0000","last_became_peered":"2024-04-25T01:08:24.497888+0000","last_unstale":"2024-04-25T01:08:29.827605+0000","last_undegraded":"2024-04-25T01:08:29.827605+0000","last_fullsized":"2024-04-25T01:08:29.827605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828026+0000","last_change":"2024-04-25T01:08:23.482207+0000","last_active":"2024-04-25T01:08:29.828026+0000","last_peered":"2024-04-25T01:08:29.828026+0000","last_clean":"2024-04-25T01:08:29.828026+0000","last_became_active":"2024-04-25T01:08:23.475806+0000","last_became_peered":"2024-04-25T01:08:23.475806+0000","last_unstale":"2024-04-25T01:08:29.828026+0000","last_undegraded":"2024-04-25T01:08:29.828026+0000","last_fullsized":"2024-04-25T01:08:29.828026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:56.775800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512324+0000","last_change":"2024-04-25T01:08:28.537642+0000","last_active":"2024-04-25T01:08:29.512324+0000","last_peered":"2024-04-25T01:08:29.512324+0000","last_clean":"2024-04-25T01:08:29.512324+0000","last_became_active":"2024-04-25T01:08:28.536434+0000","last_became_peered":"2024-04-25T01:08:28.536434+0000","last_unstale":"2024-04-25T01:08:29.512324+0000","last_undegraded":"2024-04-25T01:08:29.512324+0000","last_fullsized":"2024-04-25T01:08:29.512324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508548+0000","last_change":"2024-04-25T01:08:24.497990+0000","last_active":"2024-04-25T01:08:29.508548+0000","last_peered":"2024-04-25T01:08:29.508548+0000","last_clean":"2024-04-25T01:08:29.508548+0000","last_became_active":"2024-04-25T01:08:24.497701+0000","last_became_peered":"2024-04-25T01:08:24.497701+0000","last_unstale":"2024-04-25T01:08:29.508548+0000","last_undegraded":"2024-04-25T01:08:29.508548+0000","last_fullsized":"2024-04-25T01:08:29.508548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:25.853831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.448779+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.448310+0000","last_became_peered":"2024-04-25T01:08:23.448310+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510707+0000","last_change":"2024-04-25T01:08:28.546163+0000","last_active":"2024-04-25T01:08:29.510707+0000","last_peered":"2024-04-25T01:08:29.510707+0000","last_clean":"2024-04-25T01:08:29.510707+0000","last_became_active":"2024-04-25T01:08:28.545022+0000","last_became_peered":"2024-04-25T01:08:28.545022+0000","last_unstale":"2024-04-25T01:08:29.510707+0000","last_undegraded":"2024-04-25T01:08:29.510707+0000","last_fullsized":"2024-04-25T01:08:29.510707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508535+0000","last_change":"2024-04-25T01:08:24.472427+0000","last_active":"2024-04-25T01:08:29.508535+0000","last_peered":"2024-04-25T01:08:29.508535+0000","last_clean":"2024-04-25T01:08:29.508535+0000","last_became_active":"2024-04-25T01:08:24.471579+0000","last_became_peered":"2024-04-25T01:08:24.471579+0000","last_unstale":"2024-04-25T01:08:29.508535+0000","last_undegraded":"2024-04-25T01:08:29.508535+0000","last_fullsized":"2024-04-25T01:08:29.508535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828376+0000","last_change":"2024-04-25T01:08:23.475602+0000","last_active":"2024-04-25T01:08:29.828376+0000","last_peered":"2024-04-25T01:08:29.828376+0000","last_clean":"2024-04-25T01:08:29.828376+0000","last_became_active":"2024-04-25T01:08:23.475273+0000","last_became_peered":"2024-04-25T01:08:23.475273+0000","last_unstale":"2024-04-25T01:08:29.828376+0000","last_undegraded":"2024-04-25T01:08:29.828376+0000","last_fullsized":"2024-04-25T01:08:29.828376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830145+0000","last_change":"2024-04-25T01:08:28.858908+0000","last_active":"2024-04-25T01:08:29.830145+0000","last_peered":"2024-04-25T01:08:29.830145+0000","last_clean":"2024-04-25T01:08:29.830145+0000","last_became_active":"2024-04-25T01:08:28.858052+0000","last_became_peered":"2024-04-25T01:08:28.858052+0000","last_unstale":"2024-04-25T01:08:29.830145+0000","last_undegraded":"2024-04-25T01:08:29.830145+0000","last_fullsized":"2024-04-25T01:08:29.830145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:53.339121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511155+0000","last_change":"2024-04-25T01:08:24.473202+0000","last_active":"2024-04-25T01:08:29.511155+0000","last_peered":"2024-04-25T01:08:29.511155+0000","last_clean":"2024-04-25T01:08:29.511155+0000","last_became_active":"2024-04-25T01:08:24.470580+0000","last_became_peered":"2024-04-25T01:08:24.470580+0000","last_unstale":"2024-04-25T01:08:29.511155+0000","last_undegraded":"2024-04-25T01:08:29.511155+0000","last_fullsized":"2024-04-25T01:08:29.511155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:27.238691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828899+0000","last_change":"2024-04-25T01:08:23.482632+0000","last_active":"2024-04-25T01:08:29.828899+0000","last_peered":"2024-04-25T01:08:29.828899+0000","last_clean":"2024-04-25T01:08:29.828899+0000","last_became_active":"2024-04-25T01:08:23.482050+0000","last_became_peered":"2024-04-25T01:08:23.482050+0000","last_unstale":"2024-04-25T01:08:29.828899+0000","last_undegraded":"2024-04-25T01:08:29.828899+0000","last_fullsized":"2024-04-25T01:08:29.828899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512530+0000","last_change":"2024-04-25T01:08:28.538805+0000","last_active":"2024-04-25T01:08:29.512530+0000","last_peered":"2024-04-25T01:08:29.512530+0000","last_clean":"2024-04-25T01:08:29.512530+0000","last_became_active":"2024-04-25T01:08:28.537512+0000","last_became_peered":"2024-04-25T01:08:28.537512+0000","last_unstale":"2024-04-25T01:08:29.512530+0000","last_undegraded":"2024-04-25T01:08:29.512530+0000","last_fullsized":"2024-04-25T01:08:29.512530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829865+0000","last_change":"2024-04-25T01:08:24.488971+0000","last_active":"2024-04-25T01:08:29.829865+0000","last_peered":"2024-04-25T01:08:29.829865+0000","last_clean":"2024-04-25T01:08:29.829865+0000","last_became_active":"2024-04-25T01:08:24.488318+0000","last_became_peered":"2024-04-25T01:08:24.488318+0000","last_unstale":"2024-04-25T01:08:29.829865+0000","last_undegraded":"2024-04-25T01:08:29.829865+0000","last_fullsized":"2024-04-25T01:08:29.829865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:48.906624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829517+0000","last_change":"2024-04-25T01:08:23.484304+0000","last_active":"2024-04-25T01:08:29.829517+0000","last_peered":"2024-04-25T01:08:29.829517+0000","last_clean":"2024-04-25T01:08:29.829517+0000","last_became_active":"2024-04-25T01:08:23.483396+0000","last_became_peered":"2024-04-25T01:08:23.483396+0000","last_unstale":"2024-04-25T01:08:29.829517+0000","last_undegraded":"2024-04-25T01:08:29.829517+0000","last_fullsized":"2024-04-25T01:08:29.829517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510293+0000","last_change":"2024-04-25T01:08:28.861041+0000","last_active":"2024-04-25T01:08:29.510293+0000","last_peered":"2024-04-25T01:08:29.510293+0000","last_clean":"2024-04-25T01:08:29.510293+0000","last_became_active":"2024-04-25T01:08:28.859853+0000","last_became_peered":"2024-04-25T01:08:28.859853+0000","last_unstale":"2024-04-25T01:08:29.510293+0000","last_undegraded":"2024-04-25T01:08:29.510293+0000","last_fullsized":"2024-04-25T01:08:29.510293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:58.665747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510472+0000","last_change":"2024-04-25T01:08:24.513917+0000","last_active":"2024-04-25T01:08:29.510472+0000","last_peered":"2024-04-25T01:08:29.510472+0000","last_clean":"2024-04-25T01:08:29.510472+0000","last_became_active":"2024-04-25T01:08:24.513551+0000","last_became_peered":"2024-04-25T01:08:24.513551+0000","last_unstale":"2024-04-25T01:08:29.510472+0000","last_undegraded":"2024-04-25T01:08:29.510472+0000","last_fullsized":"2024-04-25T01:08:29.510472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829765+0000","last_change":"2024-04-25T01:08:23.463105+0000","last_active":"2024-04-25T01:08:29.829765+0000","last_peered":"2024-04-25T01:08:29.829765+0000","last_clean":"2024-04-25T01:08:29.829765+0000","last_became_active":"2024-04-25T01:08:23.462766+0000","last_became_peered":"2024-04-25T01:08:23.462766+0000","last_unstale":"2024-04-25T01:08:29.829765+0000","last_undegraded":"2024-04-25T01:08:29.829765+0000","last_fullsized":"2024-04-25T01:08:29.829765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507606+0000","last_change":"2024-04-25T01:08:28.859248+0000","last_active":"2024-04-25T01:08:29.507606+0000","last_peered":"2024-04-25T01:08:29.507606+0000","last_clean":"2024-04-25T01:08:29.507606+0000","last_became_active":"2024-04-25T01:08:28.858134+0000","last_became_peered":"2024-04-25T01:08:28.858134+0000","last_unstale":"2024-04-25T01:08:29.507606+0000","last_undegraded":"2024-04-25T01:08:29.507606+0000","last_fullsized":"2024-04-25T01:08:29.507606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:54.239410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827142+0000","last_change":"2024-04-25T01:08:24.463169+0000","last_active":"2024-04-25T01:08:29.827142+0000","last_peered":"2024-04-25T01:08:29.827142+0000","last_clean":"2024-04-25T01:08:29.827142+0000","last_became_active":"2024-04-25T01:08:24.462836+0000","last_became_peered":"2024-04-25T01:08:24.462836+0000","last_unstale":"2024-04-25T01:08:29.827142+0000","last_undegraded":"2024-04-25T01:08:29.827142+0000","last_fullsized":"2024-04-25T01:08:29.827142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826900+0000","last_change":"2024-04-25T01:08:23.482862+0000","last_active":"2024-04-25T01:08:29.826900+0000","last_peered":"2024-04-25T01:08:29.826900+0000","last_clean":"2024-04-25T01:08:29.826900+0000","last_became_active":"2024-04-25T01:08:23.482148+0000","last_became_peered":"2024-04-25T01:08:23.482148+0000","last_unstale":"2024-04-25T01:08:29.826900+0000","last_undegraded":"2024-04-25T01:08:29.826900+0000","last_fullsized":"2024-04-25T01:08:29.826900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:38.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826923+0000","last_change":"2024-04-25T01:08:28.853689+0000","last_active":"2024-04-25T01:08:29.826923+0000","last_peered":"2024-04-25T01:08:29.826923+0000","last_clean":"2024-04-25T01:08:29.826923+0000","last_became_active":"2024-04-25T01:08:28.852691+0000","last_became_peered":"2024-04-25T01:08:28.852691+0000","last_unstale":"2024-04-25T01:08:29.826923+0000","last_undegraded":"2024-04-25T01:08:29.826923+0000","last_fullsized":"2024-04-25T01:08:29.826923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:01.247919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828543+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.828543+0000","last_peered":"2024-04-25T01:08:29.828543+0000","last_clean":"2024-04-25T01:08:29.828543+0000","last_became_active":"2024-04-25T01:08:24.489833+0000","last_became_peered":"2024-04-25T01:08:24.489833+0000","last_unstale":"2024-04-25T01:08:29.828543+0000","last_undegraded":"2024-04-25T01:08:29.828543+0000","last_fullsized":"2024-04-25T01:08:29.828543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:43.847545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:23.451486+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:23.451147+0000","last_became_peered":"2024-04-25T01:08:23.451147+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824912+0000","last_change":"2024-04-25T01:08:28.546785+0000","last_active":"2024-04-25T01:08:29.824912+0000","last_peered":"2024-04-25T01:08:29.824912+0000","last_clean":"2024-04-25T01:08:29.824912+0000","last_became_active":"2024-04-25T01:08:28.546088+0000","last_became_peered":"2024-04-25T01:08:28.546088+0000","last_unstale":"2024-04-25T01:08:29.824912+0000","last_undegraded":"2024-04-25T01:08:29.824912+0000","last_fullsized":"2024-04-25T01:08:29.824912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511711+0000","last_change":"2024-04-25T01:08:24.469958+0000","last_active":"2024-04-25T01:08:29.511711+0000","last_peered":"2024-04-25T01:08:29.511711+0000","last_clean":"2024-04-25T01:08:29.511711+0000","last_became_active":"2024-04-25T01:08:24.468421+0000","last_became_peered":"2024-04-25T01:08:24.468421+0000","last_unstale":"2024-04-25T01:08:29.511711+0000","last_undegraded":"2024-04-25T01:08:29.511711+0000","last_fullsized":"2024-04-25T01:08:29.511711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:37.314427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826336+0000","last_change":"2024-04-25T01:08:23.485610+0000","last_active":"2024-04-25T01:08:29.826336+0000","last_peered":"2024-04-25T01:08:29.826336+0000","last_clean":"2024-04-25T01:08:29.826336+0000","last_became_active":"2024-04-25T01:08:23.484885+0000","last_became_peered":"2024-04-25T01:08:23.484885+0000","last_unstale":"2024-04-25T01:08:29.826336+0000","last_undegraded":"2024-04-25T01:08:29.826336+0000","last_fullsized":"2024-04-25T01:08:29.826336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:41.214012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826899+0000","last_change":"2024-04-25T01:08:28.541717+0000","last_active":"2024-04-25T01:08:29.826899+0000","last_peered":"2024-04-25T01:08:29.826899+0000","last_clean":"2024-04-25T01:08:29.826899+0000","last_became_active":"2024-04-25T01:08:28.541136+0000","last_became_peered":"2024-04-25T01:08:28.541136+0000","last_unstale":"2024-04-25T01:08:29.826899+0000","last_undegraded":"2024-04-25T01:08:29.826899+0000","last_fullsized":"2024-04-25T01:08:29.826899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824739+0000","last_change":"2024-04-25T01:08:24.468504+0000","last_active":"2024-04-25T01:08:29.824739+0000","last_peered":"2024-04-25T01:08:29.824739+0000","last_clean":"2024-04-25T01:08:29.824739+0000","last_became_active":"2024-04-25T01:08:24.468220+0000","last_became_peered":"2024-04-25T01:08:24.468220+0000","last_unstale":"2024-04-25T01:08:29.824739+0000","last_undegraded":"2024-04-25T01:08:29.824739+0000","last_fullsized":"2024-04-25T01:08:29.824739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825079+0000","last_change":"2024-04-25T01:08:23.448344+0000","last_active":"2024-04-25T01:08:29.825079+0000","last_peered":"2024-04-25T01:08:29.825079+0000","last_clean":"2024-04-25T01:08:29.825079+0000","last_became_active":"2024-04-25T01:08:23.447926+0000","last_became_peered":"2024-04-25T01:08:23.447926+0000","last_unstale":"2024-04-25T01:08:29.825079+0000","last_undegraded":"2024-04-25T01:08:29.825079+0000","last_fullsized":"2024-04-25T01:08:29.825079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830344+0000","last_change":"2024-04-25T01:08:28.547149+0000","last_active":"2024-04-25T01:08:29.830344+0000","last_peered":"2024-04-25T01:08:29.830344+0000","last_clean":"2024-04-25T01:08:29.830344+0000","last_became_active":"2024-04-25T01:08:28.546228+0000","last_became_peered":"2024-04-25T01:08:28.546228+0000","last_unstale":"2024-04-25T01:08:29.830344+0000","last_undegraded":"2024-04-25T01:08:29.830344+0000","last_fullsized":"2024-04-25T01:08:29.830344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.235046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825749+0000","last_change":"2024-04-25T01:08:24.458889+0000","last_active":"2024-04-25T01:08:29.825749+0000","last_peered":"2024-04-25T01:08:29.825749+0000","last_clean":"2024-04-25T01:08:29.825749+0000","last_became_active":"2024-04-25T01:08:24.458591+0000","last_became_peered":"2024-04-25T01:08:24.458591+0000","last_unstale":"2024-04-25T01:08:29.825749+0000","last_undegraded":"2024-04-25T01:08:29.825749+0000","last_fullsized":"2024-04-25T01:08:29.825749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:40.997594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829116+0000","last_change":"2024-04-25T01:08:23.481327+0000","last_active":"2024-04-25T01:08:29.829116+0000","last_peered":"2024-04-25T01:08:29.829116+0000","last_clean":"2024-04-25T01:08:29.829116+0000","last_became_active":"2024-04-25T01:08:23.475537+0000","last_became_peered":"2024-04-25T01:08:23.475537+0000","last_unstale":"2024-04-25T01:08:29.829116+0000","last_undegraded":"2024-04-25T01:08:29.829116+0000","last_fullsized":"2024-04-25T01:08:29.829116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829514+0000","last_change":"2024-04-25T01:08:28.859351+0000","last_active":"2024-04-25T01:08:29.829514+0000","last_peered":"2024-04-25T01:08:29.829514+0000","last_clean":"2024-04-25T01:08:29.829514+0000","last_became_active":"2024-04-25T01:08:28.858099+0000","last_became_peered":"2024-04-25T01:08:28.858099+0000","last_unstale":"2024-04-25T01:08:29.829514+0000","last_undegraded":"2024-04-25T01:08:29.829514+0000","last_fullsized":"2024-04-25T01:08:29.829514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:46.276943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508474+0000","last_change":"2024-04-25T01:08:24.495211+0000","last_active":"2024-04-25T01:08:29.508474+0000","last_peered":"2024-04-25T01:08:29.508474+0000","last_clean":"2024-04-25T01:08:29.508474+0000","last_became_active":"2024-04-25T01:08:24.494930+0000","last_became_peered":"2024-04-25T01:08:24.494930+0000","last_unstale":"2024-04-25T01:08:29.508474+0000","last_undegraded":"2024-04-25T01:08:29.508474+0000","last_fullsized":"2024-04-25T01:08:29.508474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825747+0000","last_change":"2024-04-25T01:08:23.461624+0000","last_active":"2024-04-25T01:08:29.825747+0000","last_peered":"2024-04-25T01:08:29.825747+0000","last_clean":"2024-04-25T01:08:29.825747+0000","last_became_active":"2024-04-25T01:08:23.457067+0000","last_became_peered":"2024-04-25T01:08:23.457067+0000","last_unstale":"2024-04-25T01:08:29.825747+0000","last_undegraded":"2024-04-25T01:08:29.825747+0000","last_fullsized":"2024-04-25T01:08:29.825747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829168+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:08:29.829168+0000","last_peered":"2024-04-25T01:08:29.829168+0000","last_clean":"2024-04-25T01:08:29.829168+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:08:29.829168+0000","last_undegraded":"2024-04-25T01:08:29.829168+0000","last_fullsized":"2024-04-25T01:08:29.829168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826455+0000","last_change":"2024-04-25T01:08:28.861776+0000","last_active":"2024-04-25T01:08:29.826455+0000","last_peered":"2024-04-25T01:08:29.826455+0000","last_clean":"2024-04-25T01:08:29.826455+0000","last_became_active":"2024-04-25T01:08:28.860715+0000","last_became_peered":"2024-04-25T01:08:28.860715+0000","last_unstale":"2024-04-25T01:08:29.826455+0000","last_undegraded":"2024-04-25T01:08:29.826455+0000","last_fullsized":"2024-04-25T01:08:29.826455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826011+0000","last_change":"2024-04-25T01:08:24.523631+0000","last_active":"2024-04-25T01:08:29.826011+0000","last_peered":"2024-04-25T01:08:29.826011+0000","last_clean":"2024-04-25T01:08:29.826011+0000","last_became_active":"2024-04-25T01:08:24.523130+0000","last_became_peered":"2024-04-25T01:08:24.523130+0000","last_unstale":"2024-04-25T01:08:29.826011+0000","last_undegraded":"2024-04-25T01:08:29.826011+0000","last_fullsized":"2024-04-25T01:08:29.826011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:19.861291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826698+0000","last_change":"2024-04-25T01:08:23.470284+0000","last_active":"2024-04-25T01:08:29.826698+0000","last_peered":"2024-04-25T01:08:29.826698+0000","last_clean":"2024-04-25T01:08:29.826698+0000","last_became_active":"2024-04-25T01:08:23.470043+0000","last_became_peered":"2024-04-25T01:08:23.470043+0000","last_unstale":"2024-04-25T01:08:29.826698+0000","last_undegraded":"2024-04-25T01:08:29.826698+0000","last_fullsized":"2024-04-25T01:08:29.826698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827347+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827347+0000","last_peered":"2024-04-25T01:08:29.827347+0000","last_clean":"2024-04-25T01:08:29.827347+0000","last_became_active":"2024-04-25T01:08:28.858545+0000","last_became_peered":"2024-04-25T01:08:28.858545+0000","last_unstale":"2024-04-25T01:08:29.827347+0000","last_undegraded":"2024-04-25T01:08:29.827347+0000","last_fullsized":"2024-04-25T01:08:29.827347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826582+0000","last_change":"2024-04-25T01:08:24.490983+0000","last_active":"2024-04-25T01:08:29.826582+0000","last_peered":"2024-04-25T01:08:29.826582+0000","last_clean":"2024-04-25T01:08:29.826582+0000","last_became_active":"2024-04-25T01:08:24.490276+0000","last_became_peered":"2024-04-25T01:08:24.490276+0000","last_unstale":"2024-04-25T01:08:29.826582+0000","last_undegraded":"2024-04-25T01:08:29.826582+0000","last_fullsized":"2024-04-25T01:08:29.826582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:17.470136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829394+0000","last_change":"2024-04-25T01:08:23.459773+0000","last_active":"2024-04-25T01:08:29.829394+0000","last_peered":"2024-04-25T01:08:29.829394+0000","last_clean":"2024-04-25T01:08:29.829394+0000","last_became_active":"2024-04-25T01:08:23.459374+0000","last_became_peered":"2024-04-25T01:08:23.459374+0000","last_unstale":"2024-04-25T01:08:29.829394+0000","last_undegraded":"2024-04-25T01:08:29.829394+0000","last_fullsized":"2024-04-25T01:08:29.829394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827646+0000","last_change":"2024-04-25T01:08:28.542952+0000","last_active":"2024-04-25T01:08:29.827646+0000","last_peered":"2024-04-25T01:08:29.827646+0000","last_clean":"2024-04-25T01:08:29.827646+0000","last_became_active":"2024-04-25T01:08:28.542388+0000","last_became_peered":"2024-04-25T01:08:28.542388+0000","last_unstale":"2024-04-25T01:08:29.827646+0000","last_undegraded":"2024-04-25T01:08:29.827646+0000","last_fullsized":"2024-04-25T01:08:29.827646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:47.910183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828070+0000","last_change":"2024-04-25T01:08:24.525832+0000","last_active":"2024-04-25T01:08:29.828070+0000","last_peered":"2024-04-25T01:08:29.828070+0000","last_clean":"2024-04-25T01:08:29.828070+0000","last_became_active":"2024-04-25T01:08:24.524963+0000","last_became_peered":"2024-04-25T01:08:24.524963+0000","last_unstale":"2024-04-25T01:08:29.828070+0000","last_undegraded":"2024-04-25T01:08:29.828070+0000","last_fullsized":"2024-04-25T01:08:29.828070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:54.172408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828701+0000","last_change":"2024-04-25T01:08:23.448056+0000","last_active":"2024-04-25T01:08:29.828701+0000","last_peered":"2024-04-25T01:08:29.828701+0000","last_clean":"2024-04-25T01:08:29.828701+0000","last_became_active":"2024-04-25T01:08:23.447748+0000","last_became_peered":"2024-04-25T01:08:23.447748+0000","last_unstale":"2024-04-25T01:08:29.828701+0000","last_undegraded":"2024-04-25T01:08:29.828701+0000","last_fullsized":"2024-04-25T01:08:29.828701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824750+0000","last_change":"2024-04-25T01:08:28.860380+0000","last_active":"2024-04-25T01:08:29.824750+0000","last_peered":"2024-04-25T01:08:29.824750+0000","last_clean":"2024-04-25T01:08:29.824750+0000","last_became_active":"2024-04-25T01:08:28.859396+0000","last_became_peered":"2024-04-25T01:08:28.859396+0000","last_unstale":"2024-04-25T01:08:29.824750+0000","last_undegraded":"2024-04-25T01:08:29.824750+0000","last_fullsized":"2024-04-25T01:08:29.824750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:28.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826570+0000","last_change":"2024-04-25T01:08:24.524589+0000","last_active":"2024-04-25T01:08:29.826570+0000","last_peered":"2024-04-25T01:08:29.826570+0000","last_clean":"2024-04-25T01:08:29.826570+0000","last_became_active":"2024-04-25T01:08:24.523861+0000","last_became_peered":"2024-04-25T01:08:24.523861+0000","last_unstale":"2024-04-25T01:08:29.826570+0000","last_undegraded":"2024-04-25T01:08:29.826570+0000","last_fullsized":"2024-04-25T01:08:29.826570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:53.746720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512670+0000","last_change":"2024-04-25T01:08:23.445365+0000","last_active":"2024-04-25T01:08:29.512670+0000","last_peered":"2024-04-25T01:08:29.512670+0000","last_clean":"2024-04-25T01:08:29.512670+0000","last_became_active":"2024-04-25T01:08:23.444505+0000","last_became_peered":"2024-04-25T01:08:23.444505+0000","last_unstale":"2024-04-25T01:08:29.512670+0000","last_undegraded":"2024-04-25T01:08:29.512670+0000","last_fullsized":"2024-04-25T01:08:29.512670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:51.459603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510009+0000","last_change":"2024-04-25T01:08:28.549416+0000","last_active":"2024-04-25T01:08:29.510009+0000","last_peered":"2024-04-25T01:08:29.510009+0000","last_clean":"2024-04-25T01:08:29.510009+0000","last_became_active":"2024-04-25T01:08:28.547126+0000","last_became_peered":"2024-04-25T01:08:28.547126+0000","last_unstale":"2024-04-25T01:08:29.510009+0000","last_undegraded":"2024-04-25T01:08:29.510009+0000","last_fullsized":"2024-04-25T01:08:29.510009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:29.916469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828474+0000","last_change":"2024-04-25T01:08:24.514329+0000","last_active":"2024-04-25T01:08:29.828474+0000","last_peered":"2024-04-25T01:08:29.828474+0000","last_clean":"2024-04-25T01:08:29.828474+0000","last_became_active":"2024-04-25T01:08:24.513670+0000","last_became_peered":"2024-04-25T01:08:24.513670+0000","last_unstale":"2024-04-25T01:08:29.828474+0000","last_undegraded":"2024-04-25T01:08:29.828474+0000","last_fullsized":"2024-04-25T01:08:29.828474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.823985+0000","last_change":"2024-04-25T01:08:23.449156+0000","last_active":"2024-04-25T01:08:29.823985+0000","last_peered":"2024-04-25T01:08:29.823985+0000","last_clean":"2024-04-25T01:08:29.823985+0000","last_became_active":"2024-04-25T01:08:23.448724+0000","last_became_peered":"2024-04-25T01:08:23.448724+0000","last_unstale":"2024-04-25T01:08:29.823985+0000","last_undegraded":"2024-04-25T01:08:29.823985+0000","last_fullsized":"2024-04-25T01:08:29.823985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829295+0000","last_change":"2024-04-25T01:08:28.538540+0000","last_active":"2024-04-25T01:08:29.829295+0000","last_peered":"2024-04-25T01:08:29.829295+0000","last_clean":"2024-04-25T01:08:29.829295+0000","last_became_active":"2024-04-25T01:08:28.537662+0000","last_became_peered":"2024-04-25T01:08:28.537662+0000","last_unstale":"2024-04-25T01:08:29.829295+0000","last_undegraded":"2024-04-25T01:08:29.829295+0000","last_fullsized":"2024-04-25T01:08:29.829295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:06.032667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824689+0000","last_change":"2024-04-25T01:08:24.499087+0000","last_active":"2024-04-25T01:08:29.824689+0000","last_peered":"2024-04-25T01:08:29.824689+0000","last_clean":"2024-04-25T01:08:29.824689+0000","last_became_active":"2024-04-25T01:08:24.498181+0000","last_became_peered":"2024-04-25T01:08:24.498181+0000","last_unstale":"2024-04-25T01:08:29.824689+0000","last_undegraded":"2024-04-25T01:08:29.824689+0000","last_fullsized":"2024-04-25T01:08:29.824689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825380+0000","last_change":"2024-04-25T01:08:23.466687+0000","last_active":"2024-04-25T01:08:29.825380+0000","last_peered":"2024-04-25T01:08:29.825380+0000","last_clean":"2024-04-25T01:08:29.825380+0000","last_became_active":"2024-04-25T01:08:23.465922+0000","last_became_peered":"2024-04-25T01:08:23.465922+0000","last_unstale":"2024-04-25T01:08:29.825380+0000","last_undegraded":"2024-04-25T01:08:29.825380+0000","last_fullsized":"2024-04-25T01:08:29.825380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510286+0000","last_change":"2024-04-25T01:08:28.861374+0000","last_active":"2024-04-25T01:08:29.510286+0000","last_peered":"2024-04-25T01:08:29.510286+0000","last_clean":"2024-04-25T01:08:29.510286+0000","last_became_active":"2024-04-25T01:08:28.857471+0000","last_became_peered":"2024-04-25T01:08:28.857471+0000","last_unstale":"2024-04-25T01:08:29.510286+0000","last_undegraded":"2024-04-25T01:08:29.510286+0000","last_fullsized":"2024-04-25T01:08:29.510286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:50.580633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827859+0000","last_change":"2024-04-25T01:08:24.524773+0000","last_active":"2024-04-25T01:08:29.827859+0000","last_peered":"2024-04-25T01:08:29.827859+0000","last_clean":"2024-04-25T01:08:29.827859+0000","last_became_active":"2024-04-25T01:08:24.523273+0000","last_became_peered":"2024-04-25T01:08:24.523273+0000","last_unstale":"2024-04-25T01:08:29.827859+0000","last_undegraded":"2024-04-25T01:08:29.827859+0000","last_fullsized":"2024-04-25T01:08:29.827859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:53.559257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826090+0000","last_change":"2024-04-25T01:08:23.478405+0000","last_active":"2024-04-25T01:08:29.826090+0000","last_peered":"2024-04-25T01:08:29.826090+0000","last_clean":"2024-04-25T01:08:29.826090+0000","last_became_active":"2024-04-25T01:08:23.477864+0000","last_became_peered":"2024-04-25T01:08:23.477864+0000","last_unstale":"2024-04-25T01:08:29.826090+0000","last_undegraded":"2024-04-25T01:08:29.826090+0000","last_fullsized":"2024-04-25T01:08:29.826090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:14.206151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509288+0000","last_change":"2024-04-25T01:08:28.859485+0000","last_active":"2024-04-25T01:08:29.509288+0000","last_peered":"2024-04-25T01:08:29.509288+0000","last_clean":"2024-04-25T01:08:29.509288+0000","last_became_active":"2024-04-25T01:08:28.858350+0000","last_became_peered":"2024-04-25T01:08:28.858350+0000","last_unstale":"2024-04-25T01:08:29.509288+0000","last_undegraded":"2024-04-25T01:08:29.509288+0000","last_fullsized":"2024-04-25T01:08:29.509288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:27.377063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:24.502358+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:24.502144+0000","last_became_peered":"2024-04-25T01:08:24.502144+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:17.614542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827872+0000","last_change":"2024-04-25T01:08:23.444558+0000","last_active":"2024-04-25T01:08:29.827872+0000","last_peered":"2024-04-25T01:08:29.827872+0000","last_clean":"2024-04-25T01:08:29.827872+0000","last_became_active":"2024-04-25T01:08:23.444246+0000","last_became_peered":"2024-04-25T01:08:23.444246+0000","last_unstale":"2024-04-25T01:08:29.827872+0000","last_undegraded":"2024-04-25T01:08:29.827872+0000","last_fullsized":"2024-04-25T01:08:29.827872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827499+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827499+0000","last_peered":"2024-04-25T01:08:29.827499+0000","last_clean":"2024-04-25T01:08:29.827499+0000","last_became_active":"2024-04-25T01:08:28.858324+0000","last_became_peered":"2024-04-25T01:08:28.858324+0000","last_unstale":"2024-04-25T01:08:29.827499+0000","last_undegraded":"2024-04-25T01:08:29.827499+0000","last_fullsized":"2024-04-25T01:08:29.827499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:49.074275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825751+0000","last_change":"2024-04-25T01:08:24.520516+0000","last_active":"2024-04-25T01:08:29.825751+0000","last_peered":"2024-04-25T01:08:29.825751+0000","last_clean":"2024-04-25T01:08:29.825751+0000","last_became_active":"2024-04-25T01:08:24.519967+0000","last_became_peered":"2024-04-25T01:08:24.519967+0000","last_unstale":"2024-04-25T01:08:29.825751+0000","last_undegraded":"2024-04-25T01:08:29.825751+0000","last_fullsized":"2024-04-25T01:08:29.825751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:50.176772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:35.049426+0000","last_change":"2024-04-25T01:08:23.460787+0000","last_active":"2024-04-25T01:08:35.049426+0000","last_peered":"2024-04-25T01:08:35.049426+0000","last_clean":"2024-04-25T01:08:35.049426+0000","last_became_active":"2024-04-25T01:08:23.455085+0000","last_became_peered":"2024-04-25T01:08:23.455085+0000","last_unstale":"2024-04-25T01:08:35.049426+0000","last_undegraded":"2024-04-25T01:08:35.049426+0000","last_fullsized":"2024-04-25T01:08:35.049426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825396+0000","last_change":"2024-04-25T01:08:28.536251+0000","last_active":"2024-04-25T01:08:29.825396+0000","last_peered":"2024-04-25T01:08:29.825396+0000","last_clean":"2024-04-25T01:08:29.825396+0000","last_became_active":"2024-04-25T01:08:28.535359+0000","last_became_peered":"2024-04-25T01:08:28.535359+0000","last_unstale":"2024-04-25T01:08:29.825396+0000","last_undegraded":"2024-04-25T01:08:29.825396+0000","last_fullsized":"2024-04-25T01:08:29.825396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511615+0000","last_change":"2024-04-25T01:08:24.505163+0000","last_active":"2024-04-25T01:08:29.511615+0000","last_peered":"2024-04-25T01:08:29.511615+0000","last_clean":"2024-04-25T01:08:29.511615+0000","last_became_active":"2024-04-25T01:08:24.504390+0000","last_became_peered":"2024-04-25T01:08:24.504390+0000","last_unstale":"2024-04-25T01:08:29.511615+0000","last_undegraded":"2024-04-25T01:08:29.511615+0000","last_fullsized":"2024-04-25T01:08:29.511615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829299+0000","last_change":"2024-04-25T01:08:23.484553+0000","last_active":"2024-04-25T01:08:29.829299+0000","last_peered":"2024-04-25T01:08:29.829299+0000","last_clean":"2024-04-25T01:08:29.829299+0000","last_became_active":"2024-04-25T01:08:23.484289+0000","last_became_peered":"2024-04-25T01:08:23.484289+0000","last_unstale":"2024-04-25T01:08:29.829299+0000","last_undegraded":"2024-04-25T01:08:29.829299+0000","last_fullsized":"2024-04-25T01:08:29.829299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511206+0000","last_change":"2024-04-25T01:08:28.538547+0000","last_active":"2024-04-25T01:08:29.511206+0000","last_peered":"2024-04-25T01:08:29.511206+0000","last_clean":"2024-04-25T01:08:29.511206+0000","last_became_active":"2024-04-25T01:08:28.537497+0000","last_became_peered":"2024-04-25T01:08:28.537497+0000","last_unstale":"2024-04-25T01:08:29.511206+0000","last_undegraded":"2024-04-25T01:08:29.511206+0000","last_fullsized":"2024-04-25T01:08:29.511206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830135+0000","last_change":"2024-04-25T01:08:24.493527+0000","last_active":"2024-04-25T01:08:29.830135+0000","last_peered":"2024-04-25T01:08:29.830135+0000","last_clean":"2024-04-25T01:08:29.830135+0000","last_became_active":"2024-04-25T01:08:24.493206+0000","last_became_peered":"2024-04-25T01:08:24.493206+0000","last_unstale":"2024-04-25T01:08:29.830135+0000","last_undegraded":"2024-04-25T01:08:29.830135+0000","last_fullsized":"2024-04-25T01:08:29.830135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826688+0000","last_change":"2024-04-25T01:08:23.476900+0000","last_active":"2024-04-25T01:08:29.826688+0000","last_peered":"2024-04-25T01:08:29.826688+0000","last_clean":"2024-04-25T01:08:29.826688+0000","last_became_active":"2024-04-25T01:08:23.474893+0000","last_became_peered":"2024-04-25T01:08:23.474893+0000","last_unstale":"2024-04-25T01:08:29.826688+0000","last_undegraded":"2024-04-25T01:08:29.826688+0000","last_fullsized":"2024-04-25T01:08:29.826688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:25.208026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508202+0000","last_change":"2024-04-25T01:08:28.545577+0000","last_active":"2024-04-25T01:08:29.508202+0000","last_peered":"2024-04-25T01:08:29.508202+0000","last_clean":"2024-04-25T01:08:29.508202+0000","last_became_active":"2024-04-25T01:08:28.540902+0000","last_became_peered":"2024-04-25T01:08:28.540902+0000","last_unstale":"2024-04-25T01:08:29.508202+0000","last_undegraded":"2024-04-25T01:08:29.508202+0000","last_fullsized":"2024-04-25T01:08:29.508202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:14.257846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826495+0000","last_change":"2024-04-25T01:08:24.515829+0000","last_active":"2024-04-25T01:08:29.826495+0000","last_peered":"2024-04-25T01:08:29.826495+0000","last_clean":"2024-04-25T01:08:29.826495+0000","last_became_active":"2024-04-25T01:08:24.515267+0000","last_became_peered":"2024-04-25T01:08:24.515267+0000","last_unstale":"2024-04-25T01:08:29.826495+0000","last_undegraded":"2024-04-25T01:08:29.826495+0000","last_fullsized":"2024-04-25T01:08:29.826495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:51.870029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829322+0000","last_change":"2024-04-25T01:08:23.478857+0000","last_active":"2024-04-25T01:08:29.829322+0000","last_peered":"2024-04-25T01:08:29.829322+0000","last_clean":"2024-04-25T01:08:29.829322+0000","last_became_active":"2024-04-25T01:08:23.478215+0000","last_became_peered":"2024-04-25T01:08:23.478215+0000","last_unstale":"2024-04-25T01:08:29.829322+0000","last_undegraded":"2024-04-25T01:08:29.829322+0000","last_fullsized":"2024-04-25T01:08:29.829322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508255+0000","last_change":"2024-04-25T01:08:28.863219+0000","last_active":"2024-04-25T01:08:29.508255+0000","last_peered":"2024-04-25T01:08:29.508255+0000","last_clean":"2024-04-25T01:08:29.508255+0000","last_became_active":"2024-04-25T01:08:28.862202+0000","last_became_peered":"2024-04-25T01:08:28.862202+0000","last_unstale":"2024-04-25T01:08:29.508255+0000","last_undegraded":"2024-04-25T01:08:29.508255+0000","last_fullsized":"2024-04-25T01:08:29.508255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:58.057652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829042+0000","last_change":"2024-04-25T01:08:28.864029+0000","last_active":"2024-04-25T01:08:29.829042+0000","last_peered":"2024-04-25T01:08:29.829042+0000","last_clean":"2024-04-25T01:08:29.829042+0000","last_became_active":"2024-04-25T01:08:28.863040+0000","last_became_peered":"2024-04-25T01:08:28.863040+0000","last_unstale":"2024-04-25T01:08:29.829042+0000","last_undegraded":"2024-04-25T01:08:29.829042+0000","last_fullsized":"2024-04-25T01:08:29.829042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:55.932667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827904+0000","last_change":"2024-04-25T01:08:24.472094+0000","last_active":"2024-04-25T01:08:29.827904+0000","last_peered":"2024-04-25T01:08:29.827904+0000","last_clean":"2024-04-25T01:08:29.827904+0000","last_became_active":"2024-04-25T01:08:24.471381+0000","last_became_peered":"2024-04-25T01:08:24.471381+0000","last_unstale":"2024-04-25T01:08:29.827904+0000","last_undegraded":"2024-04-25T01:08:29.827904+0000","last_fullsized":"2024-04-25T01:08:29.827904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825540+0000","last_change":"2024-04-25T01:08:23.470504+0000","last_active":"2024-04-25T01:08:29.825540+0000","last_peered":"2024-04-25T01:08:29.825540+0000","last_clean":"2024-04-25T01:08:29.825540+0000","last_became_active":"2024-04-25T01:08:23.470060+0000","last_became_peered":"2024-04-25T01:08:23.470060+0000","last_unstale":"2024-04-25T01:08:29.825540+0000","last_undegraded":"2024-04-25T01:08:29.825540+0000","last_fullsized":"2024-04-25T01:08:29.825540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:52.050718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829192+0000","last_change":"2024-04-25T01:08:28.544590+0000","last_active":"2024-04-25T01:08:29.829192+0000","last_peered":"2024-04-25T01:08:29.829192+0000","last_clean":"2024-04-25T01:08:29.829192+0000","last_became_active":"2024-04-25T01:08:28.543195+0000","last_became_peered":"2024-04-25T01:08:28.543195+0000","last_unstale":"2024-04-25T01:08:29.829192+0000","last_undegraded":"2024-04-25T01:08:29.829192+0000","last_fullsized":"2024-04-25T01:08:29.829192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508183+0000","last_change":"2024-04-25T01:08:24.494258+0000","last_active":"2024-04-25T01:08:29.508183+0000","last_peered":"2024-04-25T01:08:29.508183+0000","last_clean":"2024-04-25T01:08:29.508183+0000","last_became_active":"2024-04-25T01:08:24.493935+0000","last_became_peered":"2024-04-25T01:08:24.493935+0000","last_unstale":"2024-04-25T01:08:29.508183+0000","last_undegraded":"2024-04-25T01:08:29.508183+0000","last_fullsized":"2024-04-25T01:08:29.508183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508536+0000","last_change":"2024-04-25T01:08:23.463618+0000","last_active":"2024-04-25T01:08:29.508536+0000","last_peered":"2024-04-25T01:08:29.508536+0000","last_clean":"2024-04-25T01:08:29.508536+0000","last_became_active":"2024-04-25T01:08:23.463303+0000","last_became_peered":"2024-04-25T01:08:23.463303+0000","last_unstale":"2024-04-25T01:08:29.508536+0000","last_undegraded":"2024-04-25T01:08:29.508536+0000","last_fullsized":"2024-04-25T01:08:29.508536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828071+0000","last_change":"2024-04-25T01:08:23.465959+0000","last_active":"2024-04-25T01:08:29.828071+0000","last_peered":"2024-04-25T01:08:29.828071+0000","last_clean":"2024-04-25T01:08:29.828071+0000","last_became_active":"2024-04-25T01:08:23.465518+0000","last_became_peered":"2024-04-25T01:08:23.465518+0000","last_unstale":"2024-04-25T01:08:29.828071+0000","last_undegraded":"2024-04-25T01:08:29.828071+0000","last_fullsized":"2024-04-25T01:08:29.828071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827782+0000","last_change":"2024-04-25T01:08:24.493189+0000","last_active":"2024-04-25T01:08:29.827782+0000","last_peered":"2024-04-25T01:08:29.827782+0000","last_clean":"2024-04-25T01:08:29.827782+0000","last_became_active":"2024-04-25T01:08:24.473740+0000","last_became_peered":"2024-04-25T01:08:24.473740+0000","last_unstale":"2024-04-25T01:08:29.827782+0000","last_undegraded":"2024-04-25T01:08:29.827782+0000","last_fullsized":"2024-04-25T01:08:29.827782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825925+0000","last_change":"2024-04-25T01:08:28.542530+0000","last_active":"2024-04-25T01:08:29.825925+0000","last_peered":"2024-04-25T01:08:29.825925+0000","last_clean":"2024-04-25T01:08:29.825925+0000","last_became_active":"2024-04-25T01:08:28.541445+0000","last_became_peered":"2024-04-25T01:08:28.541445+0000","last_unstale":"2024-04-25T01:08:29.825925+0000","last_undegraded":"2024-04-25T01:08:29.825925+0000","last_fullsized":"2024-04-25T01:08:29.825925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:21.963089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508697+0000","last_change":"2024-04-25T01:08:23.475915+0000","last_active":"2024-04-25T01:08:29.508697+0000","last_peered":"2024-04-25T01:08:29.508697+0000","last_clean":"2024-04-25T01:08:29.508697+0000","last_became_active":"2024-04-25T01:08:23.475628+0000","last_became_peered":"2024-04-25T01:08:23.475628+0000","last_unstale":"2024-04-25T01:08:29.508697+0000","last_undegraded":"2024-04-25T01:08:29.508697+0000","last_fullsized":"2024-04-25T01:08:29.508697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511101+0000","last_change":"2024-04-25T01:08:24.476673+0000","last_active":"2024-04-25T01:08:29.511101+0000","last_peered":"2024-04-25T01:08:29.511101+0000","last_clean":"2024-04-25T01:08:29.511101+0000","last_became_active":"2024-04-25T01:08:24.471388+0000","last_became_peered":"2024-04-25T01:08:24.471388+0000","last_unstale":"2024-04-25T01:08:29.511101+0000","last_undegraded":"2024-04-25T01:08:29.511101+0000","last_fullsized":"2024-04-25T01:08:29.511101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:36.539667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509254+0000","last_change":"2024-04-25T01:08:28.539720+0000","last_active":"2024-04-25T01:08:29.509254+0000","last_peered":"2024-04-25T01:08:29.509254+0000","last_clean":"2024-04-25T01:08:29.509254+0000","last_became_active":"2024-04-25T01:08:28.538885+0000","last_became_peered":"2024-04-25T01:08:28.538885+0000","last_unstale":"2024-04-25T01:08:29.509254+0000","last_undegraded":"2024-04-25T01:08:29.509254+0000","last_fullsized":"2024-04-25T01:08:29.509254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:51.273909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824379+0000","last_change":"2024-04-25T01:08:24.499102+0000","last_active":"2024-04-25T01:08:29.824379+0000","last_peered":"2024-04-25T01:08:29.824379+0000","last_clean":"2024-04-25T01:08:29.824379+0000","last_became_active":"2024-04-25T01:08:24.498282+0000","last_became_peered":"2024-04-25T01:08:24.498282+0000","last_unstale":"2024-04-25T01:08:29.824379+0000","last_undegraded":"2024-04-25T01:08:29.824379+0000","last_fullsized":"2024-04-25T01:08:29.824379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824785+0000","last_change":"2024-04-25T01:08:24.477024+0000","last_active":"2024-04-25T01:08:29.824785+0000","last_peered":"2024-04-25T01:08:29.824785+0000","last_clean":"2024-04-25T01:08:29.824785+0000","last_became_active":"2024-04-25T01:08:24.476723+0000","last_became_peered":"2024-04-25T01:08:24.476723+0000","last_unstale":"2024-04-25T01:08:29.824785+0000","last_undegraded":"2024-04-25T01:08:29.824785+0000","last_fullsized":"2024-04-25T01:08:29.824785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825769+0000","last_change":"2024-04-25T01:08:24.492781+0000","last_active":"2024-04-25T01:08:29.825769+0000","last_peered":"2024-04-25T01:08:29.825769+0000","last_clean":"2024-04-25T01:08:29.825769+0000","last_became_active":"2024-04-25T01:08:24.492041+0000","last_became_peered":"2024-04-25T01:08:24.492041+0000","last_unstale":"2024-04-25T01:08:29.825769+0000","last_undegraded":"2024-04-25T01:08:29.825769+0000","last_fullsized":"2024-04-25T01:08:29.825769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:35.171404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827611+0000","last_change":"2024-04-25T01:08:24.493915+0000","last_active":"2024-04-25T01:08:29.827611+0000","last_peered":"2024-04-25T01:08:29.827611+0000","last_clean":"2024-04-25T01:08:29.827611+0000","last_became_active":"2024-04-25T01:08:24.493620+0000","last_became_peered":"2024-04-25T01:08:24.493620+0000","last_unstale":"2024-04-25T01:08:29.827611+0000","last_undegraded":"2024-04-25T01:08:29.827611+0000","last_fullsized":"2024-04-25T01:08:29.827611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:42.788724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509565+0000","last_change":"2024-04-25T01:08:24.491720+0000","last_active":"2024-04-25T01:08:29.509565+0000","last_peered":"2024-04-25T01:08:29.509565+0000","last_clean":"2024-04-25T01:08:29.509565+0000","last_became_active":"2024-04-25T01:08:24.491474+0000","last_became_peered":"2024-04-25T01:08:24.491474+0000","last_unstale":"2024-04-25T01:08:29.509565+0000","last_undegraded":"2024-04-25T01:08:29.509565+0000","last_fullsized":"2024-04-25T01:08:29.509565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828668+0000","last_change":"2024-04-25T01:08:24.472108+0000","last_active":"2024-04-25T01:08:29.828668+0000","last_peered":"2024-04-25T01:08:29.828668+0000","last_clean":"2024-04-25T01:08:29.828668+0000","last_became_active":"2024-04-25T01:08:24.471688+0000","last_became_peered":"2024-04-25T01:08:24.471688+0000","last_unstale":"2024-04-25T01:08:29.828668+0000","last_undegraded":"2024-04-25T01:08:29.828668+0000","last_fullsized":"2024-04-25T01:08:29.828668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829144+0000","last_change":"2024-04-25T01:08:24.515413+0000","last_active":"2024-04-25T01:08:29.829144+0000","last_peered":"2024-04-25T01:08:29.829144+0000","last_clean":"2024-04-25T01:08:29.829144+0000","last_became_active":"2024-04-25T01:08:24.514573+0000","last_became_peered":"2024-04-25T01:08:24.514573+0000","last_unstale":"2024-04-25T01:08:29.829144+0000","last_undegraded":"2024-04-25T01:08:29.829144+0000","last_fullsized":"2024-04-25T01:08:29.829144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510173+0000","last_change":"2024-04-25T01:08:24.487417+0000","last_active":"2024-04-25T01:08:29.510173+0000","last_peered":"2024-04-25T01:08:29.510173+0000","last_clean":"2024-04-25T01:08:29.510173+0000","last_became_active":"2024-04-25T01:08:24.487135+0000","last_became_peered":"2024-04-25T01:08:24.487135+0000","last_unstale":"2024-04-25T01:08:29.510173+0000","last_undegraded":"2024-04-25T01:08:29.510173+0000","last_fullsized":"2024-04-25T01:08:29.510173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825289+0000","last_change":"2024-04-25T01:08:24.484688+0000","last_active":"2024-04-25T01:08:29.825289+0000","last_peered":"2024-04-25T01:08:29.825289+0000","last_clean":"2024-04-25T01:08:29.825289+0000","last_became_active":"2024-04-25T01:08:24.483926+0000","last_became_peered":"2024-04-25T01:08:24.483926+0000","last_unstale":"2024-04-25T01:08:29.825289+0000","last_undegraded":"2024-04-25T01:08:29.825289+0000","last_fullsized":"2024-04-25T01:08:29.825289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508350+0000","last_change":"2024-04-25T01:08:24.487479+0000","last_active":"2024-04-25T01:08:29.508350+0000","last_peered":"2024-04-25T01:08:29.508350+0000","last_clean":"2024-04-25T01:08:29.508350+0000","last_became_active":"2024-04-25T01:08:24.487177+0000","last_became_peered":"2024-04-25T01:08:24.487177+0000","last_unstale":"2024-04-25T01:08:29.508350+0000","last_undegraded":"2024-04-25T01:08:29.508350+0000","last_fullsized":"2024-04-25T01:08:29.508350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825549+0000","last_change":"2024-04-25T01:08:24.516077+0000","last_active":"2024-04-25T01:08:29.825549+0000","last_peered":"2024-04-25T01:08:29.825549+0000","last_clean":"2024-04-25T01:08:29.825549+0000","last_became_active":"2024-04-25T01:08:24.515560+0000","last_became_peered":"2024-04-25T01:08:24.515560+0000","last_unstale":"2024-04-25T01:08:29.825549+0000","last_undegraded":"2024-04-25T01:08:29.825549+0000","last_fullsized":"2024-04-25T01:08:29.825549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:30.731050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827066+0000","last_change":"2024-04-25T01:08:24.497188+0000","last_active":"2024-04-25T01:08:29.827066+0000","last_peered":"2024-04-25T01:08:29.827066+0000","last_clean":"2024-04-25T01:08:29.827066+0000","last_became_active":"2024-04-25T01:08:24.494236+0000","last_became_peered":"2024-04-25T01:08:24.494236+0000","last_unstale":"2024-04-25T01:08:29.827066+0000","last_undegraded":"2024-04-25T01:08:29.827066+0000","last_fullsized":"2024-04-25T01:08:29.827066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:27.907903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825891+0000","last_change":"2024-04-25T01:08:24.497623+0000","last_active":"2024-04-25T01:08:29.825891+0000","last_peered":"2024-04-25T01:08:29.825891+0000","last_clean":"2024-04-25T01:08:29.825891+0000","last_became_active":"2024-04-25T01:08:24.488300+0000","last_became_peered":"2024-04-25T01:08:24.488300+0000","last_unstale":"2024-04-25T01:08:29.825891+0000","last_undegraded":"2024-04-25T01:08:29.825891+0000","last_fullsized":"2024-04-25T01:08:29.825891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826575+0000","last_change":"2024-04-25T01:08:24.525066+0000","last_active":"2024-04-25T01:08:29.826575+0000","last_peered":"2024-04-25T01:08:29.826575+0000","last_clean":"2024-04-25T01:08:29.826575+0000","last_became_active":"2024-04-25T01:08:24.524022+0000","last_became_peered":"2024-04-25T01:08:24.524022+0000","last_unstale":"2024-04-25T01:08:29.826575+0000","last_undegraded":"2024-04-25T01:08:29.826575+0000","last_fullsized":"2024-04-25T01:08:29.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:33.317294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828523+0000","last_change":"2024-04-25T01:08:24.474988+0000","last_active":"2024-04-25T01:08:29.828523+0000","last_peered":"2024-04-25T01:08:29.828523+0000","last_clean":"2024-04-25T01:08:29.828523+0000","last_became_active":"2024-04-25T01:08:24.472840+0000","last_became_peered":"2024-04-25T01:08:24.472840+0000","last_unstale":"2024-04-25T01:08:29.828523+0000","last_undegraded":"2024-04-25T01:08:29.828523+0000","last_fullsized":"2024-04-25T01:08:29.828523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827354+0000","last_change":"2024-04-25T01:08:24.504414+0000","last_active":"2024-04-25T01:08:29.827354+0000","last_peered":"2024-04-25T01:08:29.827354+0000","last_clean":"2024-04-25T01:08:29.827354+0000","last_became_active":"2024-04-25T01:08:24.503622+0000","last_became_peered":"2024-04-25T01:08:24.503622+0000","last_unstale":"2024-04-25T01:08:29.827354+0000","last_undegraded":"2024-04-25T01:08:29.827354+0000","last_fullsized":"2024-04-25T01:08:29.827354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:01.970030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825048+0000","last_change":"2024-04-25T01:08:24.496532+0000","last_active":"2024-04-25T01:08:29.825048+0000","last_peered":"2024-04-25T01:08:29.825048+0000","last_clean":"2024-04-25T01:08:29.825048+0000","last_became_active":"2024-04-25T01:08:24.495812+0000","last_became_peered":"2024-04-25T01:08:24.495812+0000","last_unstale":"2024-04-25T01:08:29.825048+0000","last_undegraded":"2024-04-25T01:08:29.825048+0000","last_fullsized":"2024-04-25T01:08:29.825048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828165+0000","last_change":"2024-04-25T01:08:24.461285+0000","last_active":"2024-04-25T01:08:29.828165+0000","last_peered":"2024-04-25T01:08:29.828165+0000","last_clean":"2024-04-25T01:08:29.828165+0000","last_became_active":"2024-04-25T01:08:24.460929+0000","last_became_peered":"2024-04-25T01:08:24.460929+0000","last_unstale":"2024-04-25T01:08:29.828165+0000","last_undegraded":"2024-04-25T01:08:29.828165+0000","last_fullsized":"2024-04-25T01:08:29.828165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827126+0000","last_change":"2024-04-25T01:08:24.493618+0000","last_active":"2024-04-25T01:08:29.827126+0000","last_peered":"2024-04-25T01:08:29.827126+0000","last_clean":"2024-04-25T01:08:29.827126+0000","last_became_active":"2024-04-25T01:08:24.489439+0000","last_became_peered":"2024-04-25T01:08:24.489439+0000","last_unstale":"2024-04-25T01:08:29.827126+0000","last_undegraded":"2024-04-25T01:08:29.827126+0000","last_fullsized":"2024-04-25T01:08:29.827126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:43.940192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508267+0000","last_change":"2024-04-25T01:08:24.478846+0000","last_active":"2024-04-25T01:08:29.508267+0000","last_peered":"2024-04-25T01:08:29.508267+0000","last_clean":"2024-04-25T01:08:29.508267+0000","last_became_active":"2024-04-25T01:08:24.478580+0000","last_became_peered":"2024-04-25T01:08:24.478580+0000","last_unstale":"2024-04-25T01:08:29.508267+0000","last_undegraded":"2024-04-25T01:08:29.508267+0000","last_fullsized":"2024-04-25T01:08:29.508267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511438+0000","last_change":"2024-04-25T01:08:24.481980+0000","last_active":"2024-04-25T01:08:29.511438+0000","last_peered":"2024-04-25T01:08:29.511438+0000","last_clean":"2024-04-25T01:08:29.511438+0000","last_became_active":"2024-04-25T01:08:24.480214+0000","last_became_peered":"2024-04-25T01:08:24.480214+0000","last_unstale":"2024-04-25T01:08:29.511438+0000","last_undegraded":"2024-04-25T01:08:29.511438+0000","last_fullsized":"2024-04-25T01:08:29.511438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826879+0000","last_change":"2024-04-25T01:08:24.480212+0000","last_active":"2024-04-25T01:08:29.826879+0000","last_peered":"2024-04-25T01:08:29.826879+0000","last_clean":"2024-04-25T01:08:29.826879+0000","last_became_active":"2024-04-25T01:08:24.477781+0000","last_became_peered":"2024-04-25T01:08:24.477781+0000","last_unstale":"2024-04-25T01:08:29.826879+0000","last_undegraded":"2024-04-25T01:08:29.826879+0000","last_fullsized":"2024-04-25T01:08:29.826879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829093+0000","last_change":"2024-04-25T01:08:24.480926+0000","last_active":"2024-04-25T01:08:29.829093+0000","last_peered":"2024-04-25T01:08:29.829093+0000","last_clean":"2024-04-25T01:08:29.829093+0000","last_became_active":"2024-04-25T01:08:24.477787+0000","last_became_peered":"2024-04-25T01:08:24.477787+0000","last_unstale":"2024-04-25T01:08:29.829093+0000","last_undegraded":"2024-04-25T01:08:29.829093+0000","last_fullsized":"2024-04-25T01:08:29.829093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827821+0000","last_change":"2024-04-25T01:08:24.525497+0000","last_active":"2024-04-25T01:08:29.827821+0000","last_peered":"2024-04-25T01:08:29.827821+0000","last_clean":"2024-04-25T01:08:29.827821+0000","last_became_active":"2024-04-25T01:08:24.524512+0000","last_became_peered":"2024-04-25T01:08:24.524512+0000","last_unstale":"2024-04-25T01:08:29.827821+0000","last_undegraded":"2024-04-25T01:08:29.827821+0000","last_fullsized":"2024-04-25T01:08:29.827821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:35.642716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507922+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.507922+0000","last_peered":"2024-04-25T01:08:29.507922+0000","last_clean":"2024-04-25T01:08:29.507922+0000","last_became_active":"2024-04-25T01:08:24.486493+0000","last_became_peered":"2024-04-25T01:08:24.486493+0000","last_unstale":"2024-04-25T01:08:29.507922+0000","last_undegraded":"2024-04-25T01:08:29.507922+0000","last_fullsized":"2024-04-25T01:08:29.507922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828202+0000","last_change":"2024-04-25T01:08:24.510776+0000","last_active":"2024-04-25T01:08:29.828202+0000","last_peered":"2024-04-25T01:08:29.828202+0000","last_clean":"2024-04-25T01:08:29.828202+0000","last_became_active":"2024-04-25T01:08:24.509980+0000","last_became_peered":"2024-04-25T01:08:24.509980+0000","last_unstale":"2024-04-25T01:08:29.828202+0000","last_undegraded":"2024-04-25T01:08:29.828202+0000","last_fullsized":"2024-04-25T01:08:29.828202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825321+0000","last_change":"2024-04-25T01:08:24.497109+0000","last_active":"2024-04-25T01:08:29.825321+0000","last_peered":"2024-04-25T01:08:29.825321+0000","last_clean":"2024-04-25T01:08:29.825321+0000","last_became_active":"2024-04-25T01:08:24.496213+0000","last_became_peered":"2024-04-25T01:08:24.496213+0000","last_unstale":"2024-04-25T01:08:29.825321+0000","last_undegraded":"2024-04-25T01:08:29.825321+0000","last_fullsized":"2024-04-25T01:08:29.825321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827021+0000","last_change":"2024-04-25T01:08:24.524338+0000","last_active":"2024-04-25T01:08:29.827021+0000","last_peered":"2024-04-25T01:08:29.827021+0000","last_clean":"2024-04-25T01:08:29.827021+0000","last_became_active":"2024-04-25T01:08:24.523936+0000","last_became_peered":"2024-04-25T01:08:24.523936+0000","last_unstale":"2024-04-25T01:08:29.827021+0000","last_undegraded":"2024-04-25T01:08:29.827021+0000","last_fullsized":"2024-04-25T01:08:29.827021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:50.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826302+0000","last_change":"2024-04-25T01:08:24.491543+0000","last_active":"2024-04-25T01:08:29.826302+0000","last_peered":"2024-04-25T01:08:29.826302+0000","last_clean":"2024-04-25T01:08:29.826302+0000","last_became_active":"2024-04-25T01:08:24.490866+0000","last_became_peered":"2024-04-25T01:08:24.490866+0000","last_unstale":"2024-04-25T01:08:29.826302+0000","last_undegraded":"2024-04-25T01:08:29.826302+0000","last_fullsized":"2024-04-25T01:08:29.826302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:38.093629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828785+0000","last_change":"2024-04-25T01:08:24.459335+0000","last_active":"2024-04-25T01:08:29.828785+0000","last_peered":"2024-04-25T01:08:29.828785+0000","last_clean":"2024-04-25T01:08:29.828785+0000","last_became_active":"2024-04-25T01:08:24.458795+0000","last_became_peered":"2024-04-25T01:08:24.458795+0000","last_unstale":"2024-04-25T01:08:29.828785+0000","last_undegraded":"2024-04-25T01:08:29.828785+0000","last_fullsized":"2024-04-25T01:08:29.828785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:10.682483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829652+0000","last_change":"2024-04-25T01:08:24.465465+0000","last_active":"2024-04-25T01:08:29.829652+0000","last_peered":"2024-04-25T01:08:29.829652+0000","last_clean":"2024-04-25T01:08:29.829652+0000","last_became_active":"2024-04-25T01:08:24.464433+0000","last_became_peered":"2024-04-25T01:08:24.464433+0000","last_unstale":"2024-04-25T01:08:29.829652+0000","last_undegraded":"2024-04-25T01:08:29.829652+0000","last_fullsized":"2024-04-25T01:08:29.829652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827333+0000","last_change":"2024-04-25T01:08:24.525387+0000","last_active":"2024-04-25T01:08:29.827333+0000","last_peered":"2024-04-25T01:08:29.827333+0000","last_clean":"2024-04-25T01:08:29.827333+0000","last_became_active":"2024-04-25T01:08:24.524324+0000","last_became_peered":"2024-04-25T01:08:24.524324+0000","last_unstale":"2024-04-25T01:08:29.827333+0000","last_undegraded":"2024-04-25T01:08:29.827333+0000","last_fullsized":"2024-04-25T01:08:29.827333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:49.276483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":102463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":102553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":693469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:08:46.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:08:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c5d877640 1 Processor -- start 2024-04-25T01:08:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c5d877640 1 -- start start 2024-04-25T01:08:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c5d877640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c58145470 0x7f4c5815c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c5d877640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c580b5530 0x7f4c580cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c5d877640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c5815f9a0 con 0x7f4c580cf0e0 2024-04-25T01:08:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c5d877640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f4c5815f750 con 0x7f4c580b5530 2024-04-25T01:08:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c5d877640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5815fc80 con 0x7f4c58145470 2024-04-25T01:08:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c577fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c580b5530 0x7f4c580cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c577fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f4c580b5530 0x7f4c580cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50258/0 (socket says 172.21.15.33:50258) 2024-04-25T01:08:46.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c577fe640 1 -- 172.21.15.33:0/2777972242 learned_addr learned my addr 172.21.15.33:0/2777972242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:46.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c56ffd640 1 --2- 172.21.15.33:0/2777972242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c58145470 0x7f4c5815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c577fe640 1 -- 172.21.15.33:0/2777972242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c58145470 msgr2=0x7f4c5815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:46.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c577fe640 1 --2- 172.21.15.33:0/2777972242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c58145470 0x7f4c5815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c577fe640 1 -- 172.21.15.33:0/2777972242 >> v1:172.21.15.138:6789/0 conn(0x7f4c580cf0e0 legacy=0x7f4c580b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c577fe640 1 -- 172.21.15.33:0/2777972242 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5815ff60 con 0x7f4c580b5530 2024-04-25T01:08:46.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.705+0000 7f4c56ffd640 1 --2- 172.21.15.33:0/2777972242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c58145470 0x7f4c5815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:46.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c577fe640 1 --2- 172.21.15.33:0/2777972242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c580b5530 0x7f4c580cce00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009e50 tx=0x7f4c5001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fa902abe356ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c567fc640 1 -- 172.21.15.33:0/2777972242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500a5810 con 0x7f4c580b5530 2024-04-25T01:08:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c567fc640 1 -- 172.21.15.33:0/2777972242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c500a59b0 con 0x7f4c580b5530 2024-04-25T01:08:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c567fc640 1 -- 172.21.15.33:0/2777972242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500a5d80 con 0x7f4c580b5530 2024-04-25T01:08:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 -- 172.21.15.33:0/2777972242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c580b5530 msgr2=0x7f4c580cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 --2- 172.21.15.33:0/2777972242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c580b5530 0x7f4c580cce00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009e50 tx=0x7f4c5001ca30 comp rx=0 tx=0).stop 2024-04-25T01:08:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 -- 172.21.15.33:0/2777972242 shutdown_connections 2024-04-25T01:08:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 --2- 172.21.15.33:0/2777972242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c580b5530 0x7f4c580cce00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 --2- 172.21.15.33:0/2777972242 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c58145470 0x7f4c5815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 -- 172.21.15.33:0/2777972242 >> 172.21.15.33:0/2777972242 conn(0x7f4c580b1810 msgr2=0x7f4c580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 -- 172.21.15.33:0/2777972242 shutdown_connections 2024-04-25T01:08:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.709+0000 7f4c5d877640 1 -- 172.21.15.33:0/2777972242 wait complete. 2024-04-25T01:08:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 Processor -- start 2024-04-25T01:08:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 -- start start 2024-04-25T01:08:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c580b5530 0x7f4c58141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c580cf0e0 0x7f4c5813aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c58145470 0x7f4c5813fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4c58168c70 con 0x7f4c58145470 2024-04-25T01:08:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c58168af0 con 0x7f4c580b5530 2024-04-25T01:08:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4c58168df0 con 0x7f4c580cf0e0 2024-04-25T01:08:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c58145470 0x7f4c5813fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c58145470 0x7f4c5813fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50268/0 (socket says 172.21.15.33:50268) 2024-04-25T01:08:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 -- 172.21.15.33:0/975082503 learned_addr learned my addr 172.21.15.33:0/975082503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c56ffd640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c580b5530 0x7f4c58141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c4ffff640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c580cf0e0 0x7f4c5813aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 -- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c580cf0e0 msgr2=0x7f4c5813aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c580cf0e0 0x7f4c5813aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 -- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c580b5530 msgr2=0x7f4c58141c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c580b5530 0x7f4c58141c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 -- 172.21.15.33:0/975082503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c58154ac0 con 0x7f4c58145470 2024-04-25T01:08:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c577fe640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c58145470 0x7f4c5813fbf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4c500af040 tx=0x7f4c500197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c54ff9640 1 -- 172.21.15.33:0/975082503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500bc020 con 0x7f4c58145470 2024-04-25T01:08:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c54ff9640 1 -- 172.21.15.33:0/975082503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c500b8670 con 0x7f4c58145470 2024-04-25T01:08:46.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c58154da0 con 0x7f4c58145470 2024-04-25T01:08:46.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c54ff9640 1 -- 172.21.15.33:0/975082503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c500b8a20 con 0x7f4c58145470 2024-04-25T01:08:46.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c58155360 con 0x7f4c58145470 2024-04-25T01:08:46.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c58069040 con 0x7f4c58145470 2024-04-25T01:08:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.713+0000 7f4c54ff9640 1 -- 172.21.15.33:0/975082503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4c500bf020 con 0x7f4c58145470 2024-04-25T01:08:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.717+0000 7f4c54ff9640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4c380678c0 0x7f4c38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.717+0000 7f4c54ff9640 1 -- 172.21.15.33:0/975082503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4c50120fe0 con 0x7f4c58145470 2024-04-25T01:08:46.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.717+0000 7f4c56ffd640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4c380678c0 0x7f4c38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:46.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.717+0000 7f4c56ffd640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4c380678c0 0x7f4c38069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c5806e730 tx=0x7f4c48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:46.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.721+0000 7f4c54ff9640 1 -- 172.21.15.33:0/975082503 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4c500c6030 con 0x7f4c58145470 2024-04-25T01:08:46.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.869+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4c58153d40 con 0x7f4c380678c0 2024-04-25T01:08:46.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.893+0000 7f4c54ff9640 1 -- 172.21.15.33:0/975082503 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f4c58153d40 con 0x7f4c380678c0 2024-04-25T01:08:46.889 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:46.891 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:08:46.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4c380678c0 msgr2=0x7f4c38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4c380678c0 0x7f4c38069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c5806e730 tx=0x7f4c48008040 comp rx=0 tx=0).stop 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c58145470 msgr2=0x7f4c5813fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c58145470 0x7f4c5813fbf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4c500af040 tx=0x7f4c500197d0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 shutdown_connections 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f4c380678c0 0x7f4c38069d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c58145470 0x7f4c5813fbf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f4c580cf0e0 0x7f4c5813aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 --2- 172.21.15.33:0/975082503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c580b5530 0x7f4c58141c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 >> 172.21.15.33:0/975082503 conn(0x7f4c580b1810 msgr2=0x7f4c580b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 shutdown_connections 2024-04-25T01:08:46.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:46.897+0000 7f4c5d877640 1 -- 172.21.15.33:0/975082503 wait complete. 2024-04-25T01:08:46.906 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T01:08:45.092884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223588,"kb_used_data":3240,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751132,"statfs":{"total":773094113280,"available":772865159168,"internally_reserved":0,"allocated":3317760,"data_stored":2009720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005168"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830172+0000","last_change":"2024-04-25T01:08:23.464284+0000","last_active":"2024-04-25T01:08:29.830172+0000","last_peered":"2024-04-25T01:08:29.830172+0000","last_clean":"2024-04-25T01:08:29.830172+0000","last_became_active":"2024-04-25T01:08:23.464018+0000","last_became_peered":"2024-04-25T01:08:23.464018+0000","last_unstale":"2024-04-25T01:08:29.830172+0000","last_undegraded":"2024-04-25T01:08:29.830172+0000","last_fullsized":"2024-04-25T01:08:29.830172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:10:27.195443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509770+0000","last_change":"2024-04-25T01:08:24.479420+0000","last_active":"2024-04-25T01:08:29.509770+0000","last_peered":"2024-04-25T01:08:29.509770+0000","last_clean":"2024-04-25T01:08:29.509770+0000","last_became_active":"2024-04-25T01:08:24.479003+0000","last_became_peered":"2024-04-25T01:08:24.479003+0000","last_unstale":"2024-04-25T01:08:29.509770+0000","last_undegraded":"2024-04-25T01:08:29.509770+0000","last_fullsized":"2024-04-25T01:08:29.509770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:10.309810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507734+0000","last_change":"2024-04-25T01:08:28.865743+0000","last_active":"2024-04-25T01:08:29.507734+0000","last_peered":"2024-04-25T01:08:29.507734+0000","last_clean":"2024-04-25T01:08:29.507734+0000","last_became_active":"2024-04-25T01:08:28.864434+0000","last_became_peered":"2024-04-25T01:08:28.864434+0000","last_unstale":"2024-04-25T01:08:29.507734+0000","last_undegraded":"2024-04-25T01:08:29.507734+0000","last_fullsized":"2024-04-25T01:08:29.507734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:37.379931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510114+0000","last_change":"2024-04-25T01:08:23.447876+0000","last_active":"2024-04-25T01:08:29.510114+0000","last_peered":"2024-04-25T01:08:29.510114+0000","last_clean":"2024-04-25T01:08:29.510114+0000","last_became_active":"2024-04-25T01:08:23.447588+0000","last_became_peered":"2024-04-25T01:08:23.447588+0000","last_unstale":"2024-04-25T01:08:29.510114+0000","last_undegraded":"2024-04-25T01:08:29.510114+0000","last_fullsized":"2024-04-25T01:08:29.510114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830021+0000","last_change":"2024-04-25T01:08:24.494422+0000","last_active":"2024-04-25T01:08:29.830021+0000","last_peered":"2024-04-25T01:08:29.830021+0000","last_clean":"2024-04-25T01:08:29.830021+0000","last_became_active":"2024-04-25T01:08:24.494035+0000","last_became_peered":"2024-04-25T01:08:24.494035+0000","last_unstale":"2024-04-25T01:08:29.830021+0000","last_undegraded":"2024-04-25T01:08:29.830021+0000","last_fullsized":"2024-04-25T01:08:29.830021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510126+0000","last_change":"2024-04-25T01:08:28.548777+0000","last_active":"2024-04-25T01:08:29.510126+0000","last_peered":"2024-04-25T01:08:29.510126+0000","last_clean":"2024-04-25T01:08:29.510126+0000","last_became_active":"2024-04-25T01:08:28.545213+0000","last_became_peered":"2024-04-25T01:08:28.545213+0000","last_unstale":"2024-04-25T01:08:29.510126+0000","last_undegraded":"2024-04-25T01:08:29.510126+0000","last_fullsized":"2024-04-25T01:08:29.510126+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:41:14.372448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826296+0000","last_change":"2024-04-25T01:08:23.483744+0000","last_active":"2024-04-25T01:08:29.826296+0000","last_peered":"2024-04-25T01:08:29.826296+0000","last_clean":"2024-04-25T01:08:29.826296+0000","last_became_active":"2024-04-25T01:08:23.483245+0000","last_became_peered":"2024-04-25T01:08:23.483245+0000","last_unstale":"2024-04-25T01:08:29.826296+0000","last_undegraded":"2024-04-25T01:08:29.826296+0000","last_fullsized":"2024-04-25T01:08:29.826296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:38.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826404+0000","last_change":"2024-04-25T01:08:24.480206+0000","last_active":"2024-04-25T01:08:29.826404+0000","last_peered":"2024-04-25T01:08:29.826404+0000","last_clean":"2024-04-25T01:08:29.826404+0000","last_became_active":"2024-04-25T01:08:24.479327+0000","last_became_peered":"2024-04-25T01:08:24.479327+0000","last_unstale":"2024-04-25T01:08:29.826404+0000","last_undegraded":"2024-04-25T01:08:29.826404+0000","last_fullsized":"2024-04-25T01:08:29.826404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:30:19.885275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510631+0000","last_change":"2024-04-25T01:08:28.545701+0000","last_active":"2024-04-25T01:08:29.510631+0000","last_peered":"2024-04-25T01:08:29.510631+0000","last_clean":"2024-04-25T01:08:29.510631+0000","last_became_active":"2024-04-25T01:08:28.544731+0000","last_became_peered":"2024-04-25T01:08:28.544731+0000","last_unstale":"2024-04-25T01:08:29.510631+0000","last_undegraded":"2024-04-25T01:08:29.510631+0000","last_fullsized":"2024-04-25T01:08:29.510631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.483124+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.482403+0000","last_became_peered":"2024-04-25T01:08:23.482403+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:22.095395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827570+0000","last_change":"2024-04-25T01:08:24.488697+0000","last_active":"2024-04-25T01:08:29.827570+0000","last_peered":"2024-04-25T01:08:29.827570+0000","last_clean":"2024-04-25T01:08:29.827570+0000","last_became_active":"2024-04-25T01:08:24.487850+0000","last_became_peered":"2024-04-25T01:08:24.487850+0000","last_unstale":"2024-04-25T01:08:29.827570+0000","last_undegraded":"2024-04-25T01:08:29.827570+0000","last_fullsized":"2024-04-25T01:08:29.827570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:57.185281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510492+0000","last_change":"2024-04-25T01:08:28.545916+0000","last_active":"2024-04-25T01:08:29.510492+0000","last_peered":"2024-04-25T01:08:29.510492+0000","last_clean":"2024-04-25T01:08:29.510492+0000","last_became_active":"2024-04-25T01:08:28.544666+0000","last_became_peered":"2024-04-25T01:08:28.544666+0000","last_unstale":"2024-04-25T01:08:29.510492+0000","last_undegraded":"2024-04-25T01:08:29.510492+0000","last_fullsized":"2024-04-25T01:08:29.510492+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:03.657733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827605+0000","last_change":"2024-04-25T01:08:24.498408+0000","last_active":"2024-04-25T01:08:29.827605+0000","last_peered":"2024-04-25T01:08:29.827605+0000","last_clean":"2024-04-25T01:08:29.827605+0000","last_became_active":"2024-04-25T01:08:24.497888+0000","last_became_peered":"2024-04-25T01:08:24.497888+0000","last_unstale":"2024-04-25T01:08:29.827605+0000","last_undegraded":"2024-04-25T01:08:29.827605+0000","last_fullsized":"2024-04-25T01:08:29.827605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828026+0000","last_change":"2024-04-25T01:08:23.482207+0000","last_active":"2024-04-25T01:08:29.828026+0000","last_peered":"2024-04-25T01:08:29.828026+0000","last_clean":"2024-04-25T01:08:29.828026+0000","last_became_active":"2024-04-25T01:08:23.475806+0000","last_became_peered":"2024-04-25T01:08:23.475806+0000","last_unstale":"2024-04-25T01:08:29.828026+0000","last_undegraded":"2024-04-25T01:08:29.828026+0000","last_fullsized":"2024-04-25T01:08:29.828026+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:25:56.775800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512324+0000","last_change":"2024-04-25T01:08:28.537642+0000","last_active":"2024-04-25T01:08:29.512324+0000","last_peered":"2024-04-25T01:08:29.512324+0000","last_clean":"2024-04-25T01:08:29.512324+0000","last_became_active":"2024-04-25T01:08:28.536434+0000","last_became_peered":"2024-04-25T01:08:28.536434+0000","last_unstale":"2024-04-25T01:08:29.512324+0000","last_undegraded":"2024-04-25T01:08:29.512324+0000","last_fullsized":"2024-04-25T01:08:29.512324+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508548+0000","last_change":"2024-04-25T01:08:24.497990+0000","last_active":"2024-04-25T01:08:29.508548+0000","last_peered":"2024-04-25T01:08:29.508548+0000","last_clean":"2024-04-25T01:08:29.508548+0000","last_became_active":"2024-04-25T01:08:24.497701+0000","last_became_peered":"2024-04-25T01:08:24.497701+0000","last_unstale":"2024-04-25T01:08:29.508548+0000","last_undegraded":"2024-04-25T01:08:29.508548+0000","last_fullsized":"2024-04-25T01:08:29.508548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:25.853831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825949+0000","last_change":"2024-04-25T01:08:23.448779+0000","last_active":"2024-04-25T01:08:29.825949+0000","last_peered":"2024-04-25T01:08:29.825949+0000","last_clean":"2024-04-25T01:08:29.825949+0000","last_became_active":"2024-04-25T01:08:23.448310+0000","last_became_peered":"2024-04-25T01:08:23.448310+0000","last_unstale":"2024-04-25T01:08:29.825949+0000","last_undegraded":"2024-04-25T01:08:29.825949+0000","last_fullsized":"2024-04-25T01:08:29.825949+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510707+0000","last_change":"2024-04-25T01:08:28.546163+0000","last_active":"2024-04-25T01:08:29.510707+0000","last_peered":"2024-04-25T01:08:29.510707+0000","last_clean":"2024-04-25T01:08:29.510707+0000","last_became_active":"2024-04-25T01:08:28.545022+0000","last_became_peered":"2024-04-25T01:08:28.545022+0000","last_unstale":"2024-04-25T01:08:29.510707+0000","last_undegraded":"2024-04-25T01:08:29.510707+0000","last_fullsized":"2024-04-25T01:08:29.510707+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508535+0000","last_change":"2024-04-25T01:08:24.472427+0000","last_active":"2024-04-25T01:08:29.508535+0000","last_peered":"2024-04-25T01:08:29.508535+0000","last_clean":"2024-04-25T01:08:29.508535+0000","last_became_active":"2024-04-25T01:08:24.471579+0000","last_became_peered":"2024-04-25T01:08:24.471579+0000","last_unstale":"2024-04-25T01:08:29.508535+0000","last_undegraded":"2024-04-25T01:08:29.508535+0000","last_fullsized":"2024-04-25T01:08:29.508535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828376+0000","last_change":"2024-04-25T01:08:23.475602+0000","last_active":"2024-04-25T01:08:29.828376+0000","last_peered":"2024-04-25T01:08:29.828376+0000","last_clean":"2024-04-25T01:08:29.828376+0000","last_became_active":"2024-04-25T01:08:23.475273+0000","last_became_peered":"2024-04-25T01:08:23.475273+0000","last_unstale":"2024-04-25T01:08:29.828376+0000","last_undegraded":"2024-04-25T01:08:29.828376+0000","last_fullsized":"2024-04-25T01:08:29.828376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830145+0000","last_change":"2024-04-25T01:08:28.858908+0000","last_active":"2024-04-25T01:08:29.830145+0000","last_peered":"2024-04-25T01:08:29.830145+0000","last_clean":"2024-04-25T01:08:29.830145+0000","last_became_active":"2024-04-25T01:08:28.858052+0000","last_became_peered":"2024-04-25T01:08:28.858052+0000","last_unstale":"2024-04-25T01:08:29.830145+0000","last_undegraded":"2024-04-25T01:08:29.830145+0000","last_fullsized":"2024-04-25T01:08:29.830145+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:53.339121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511155+0000","last_change":"2024-04-25T01:08:24.473202+0000","last_active":"2024-04-25T01:08:29.511155+0000","last_peered":"2024-04-25T01:08:29.511155+0000","last_clean":"2024-04-25T01:08:29.511155+0000","last_became_active":"2024-04-25T01:08:24.470580+0000","last_became_peered":"2024-04-25T01:08:24.470580+0000","last_unstale":"2024-04-25T01:08:29.511155+0000","last_undegraded":"2024-04-25T01:08:29.511155+0000","last_fullsized":"2024-04-25T01:08:29.511155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:27.238691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828899+0000","last_change":"2024-04-25T01:08:23.482632+0000","last_active":"2024-04-25T01:08:29.828899+0000","last_peered":"2024-04-25T01:08:29.828899+0000","last_clean":"2024-04-25T01:08:29.828899+0000","last_became_active":"2024-04-25T01:08:23.482050+0000","last_became_peered":"2024-04-25T01:08:23.482050+0000","last_unstale":"2024-04-25T01:08:29.828899+0000","last_undegraded":"2024-04-25T01:08:29.828899+0000","last_fullsized":"2024-04-25T01:08:29.828899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512530+0000","last_change":"2024-04-25T01:08:28.538805+0000","last_active":"2024-04-25T01:08:29.512530+0000","last_peered":"2024-04-25T01:08:29.512530+0000","last_clean":"2024-04-25T01:08:29.512530+0000","last_became_active":"2024-04-25T01:08:28.537512+0000","last_became_peered":"2024-04-25T01:08:28.537512+0000","last_unstale":"2024-04-25T01:08:29.512530+0000","last_undegraded":"2024-04-25T01:08:29.512530+0000","last_fullsized":"2024-04-25T01:08:29.512530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829865+0000","last_change":"2024-04-25T01:08:24.488971+0000","last_active":"2024-04-25T01:08:29.829865+0000","last_peered":"2024-04-25T01:08:29.829865+0000","last_clean":"2024-04-25T01:08:29.829865+0000","last_became_active":"2024-04-25T01:08:24.488318+0000","last_became_peered":"2024-04-25T01:08:24.488318+0000","last_unstale":"2024-04-25T01:08:29.829865+0000","last_undegraded":"2024-04-25T01:08:29.829865+0000","last_fullsized":"2024-04-25T01:08:29.829865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:48.906624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829517+0000","last_change":"2024-04-25T01:08:23.484304+0000","last_active":"2024-04-25T01:08:29.829517+0000","last_peered":"2024-04-25T01:08:29.829517+0000","last_clean":"2024-04-25T01:08:29.829517+0000","last_became_active":"2024-04-25T01:08:23.483396+0000","last_became_peered":"2024-04-25T01:08:23.483396+0000","last_unstale":"2024-04-25T01:08:29.829517+0000","last_undegraded":"2024-04-25T01:08:29.829517+0000","last_fullsized":"2024-04-25T01:08:29.829517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510293+0000","last_change":"2024-04-25T01:08:28.861041+0000","last_active":"2024-04-25T01:08:29.510293+0000","last_peered":"2024-04-25T01:08:29.510293+0000","last_clean":"2024-04-25T01:08:29.510293+0000","last_became_active":"2024-04-25T01:08:28.859853+0000","last_became_peered":"2024-04-25T01:08:28.859853+0000","last_unstale":"2024-04-25T01:08:29.510293+0000","last_undegraded":"2024-04-25T01:08:29.510293+0000","last_fullsized":"2024-04-25T01:08:29.510293+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:58.665747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510472+0000","last_change":"2024-04-25T01:08:24.513917+0000","last_active":"2024-04-25T01:08:29.510472+0000","last_peered":"2024-04-25T01:08:29.510472+0000","last_clean":"2024-04-25T01:08:29.510472+0000","last_became_active":"2024-04-25T01:08:24.513551+0000","last_became_peered":"2024-04-25T01:08:24.513551+0000","last_unstale":"2024-04-25T01:08:29.510472+0000","last_undegraded":"2024-04-25T01:08:29.510472+0000","last_fullsized":"2024-04-25T01:08:29.510472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829765+0000","last_change":"2024-04-25T01:08:23.463105+0000","last_active":"2024-04-25T01:08:29.829765+0000","last_peered":"2024-04-25T01:08:29.829765+0000","last_clean":"2024-04-25T01:08:29.829765+0000","last_became_active":"2024-04-25T01:08:23.462766+0000","last_became_peered":"2024-04-25T01:08:23.462766+0000","last_unstale":"2024-04-25T01:08:29.829765+0000","last_undegraded":"2024-04-25T01:08:29.829765+0000","last_fullsized":"2024-04-25T01:08:29.829765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507606+0000","last_change":"2024-04-25T01:08:28.859248+0000","last_active":"2024-04-25T01:08:29.507606+0000","last_peered":"2024-04-25T01:08:29.507606+0000","last_clean":"2024-04-25T01:08:29.507606+0000","last_became_active":"2024-04-25T01:08:28.858134+0000","last_became_peered":"2024-04-25T01:08:28.858134+0000","last_unstale":"2024-04-25T01:08:29.507606+0000","last_undegraded":"2024-04-25T01:08:29.507606+0000","last_fullsized":"2024-04-25T01:08:29.507606+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:54.239410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827142+0000","last_change":"2024-04-25T01:08:24.463169+0000","last_active":"2024-04-25T01:08:29.827142+0000","last_peered":"2024-04-25T01:08:29.827142+0000","last_clean":"2024-04-25T01:08:29.827142+0000","last_became_active":"2024-04-25T01:08:24.462836+0000","last_became_peered":"2024-04-25T01:08:24.462836+0000","last_unstale":"2024-04-25T01:08:29.827142+0000","last_undegraded":"2024-04-25T01:08:29.827142+0000","last_fullsized":"2024-04-25T01:08:29.827142+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826900+0000","last_change":"2024-04-25T01:08:23.482862+0000","last_active":"2024-04-25T01:08:29.826900+0000","last_peered":"2024-04-25T01:08:29.826900+0000","last_clean":"2024-04-25T01:08:29.826900+0000","last_became_active":"2024-04-25T01:08:23.482148+0000","last_became_peered":"2024-04-25T01:08:23.482148+0000","last_unstale":"2024-04-25T01:08:29.826900+0000","last_undegraded":"2024-04-25T01:08:29.826900+0000","last_fullsized":"2024-04-25T01:08:29.826900+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:38.361553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826923+0000","last_change":"2024-04-25T01:08:28.853689+0000","last_active":"2024-04-25T01:08:29.826923+0000","last_peered":"2024-04-25T01:08:29.826923+0000","last_clean":"2024-04-25T01:08:29.826923+0000","last_became_active":"2024-04-25T01:08:28.852691+0000","last_became_peered":"2024-04-25T01:08:28.852691+0000","last_unstale":"2024-04-25T01:08:29.826923+0000","last_undegraded":"2024-04-25T01:08:29.826923+0000","last_fullsized":"2024-04-25T01:08:29.826923+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:19:01.247919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828543+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.828543+0000","last_peered":"2024-04-25T01:08:29.828543+0000","last_clean":"2024-04-25T01:08:29.828543+0000","last_became_active":"2024-04-25T01:08:24.489833+0000","last_became_peered":"2024-04-25T01:08:24.489833+0000","last_unstale":"2024-04-25T01:08:29.828543+0000","last_undegraded":"2024-04-25T01:08:29.828543+0000","last_fullsized":"2024-04-25T01:08:29.828543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:43.847545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:23.451486+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:23.451147+0000","last_became_peered":"2024-04-25T01:08:23.451147+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824912+0000","last_change":"2024-04-25T01:08:28.546785+0000","last_active":"2024-04-25T01:08:29.824912+0000","last_peered":"2024-04-25T01:08:29.824912+0000","last_clean":"2024-04-25T01:08:29.824912+0000","last_became_active":"2024-04-25T01:08:28.546088+0000","last_became_peered":"2024-04-25T01:08:28.546088+0000","last_unstale":"2024-04-25T01:08:29.824912+0000","last_undegraded":"2024-04-25T01:08:29.824912+0000","last_fullsized":"2024-04-25T01:08:29.824912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511711+0000","last_change":"2024-04-25T01:08:24.469958+0000","last_active":"2024-04-25T01:08:29.511711+0000","last_peered":"2024-04-25T01:08:29.511711+0000","last_clean":"2024-04-25T01:08:29.511711+0000","last_became_active":"2024-04-25T01:08:24.468421+0000","last_became_peered":"2024-04-25T01:08:24.468421+0000","last_unstale":"2024-04-25T01:08:29.511711+0000","last_undegraded":"2024-04-25T01:08:29.511711+0000","last_fullsized":"2024-04-25T01:08:29.511711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:37.314427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826336+0000","last_change":"2024-04-25T01:08:23.485610+0000","last_active":"2024-04-25T01:08:29.826336+0000","last_peered":"2024-04-25T01:08:29.826336+0000","last_clean":"2024-04-25T01:08:29.826336+0000","last_became_active":"2024-04-25T01:08:23.484885+0000","last_became_peered":"2024-04-25T01:08:23.484885+0000","last_unstale":"2024-04-25T01:08:29.826336+0000","last_undegraded":"2024-04-25T01:08:29.826336+0000","last_fullsized":"2024-04-25T01:08:29.826336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:32:41.214012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826899+0000","last_change":"2024-04-25T01:08:28.541717+0000","last_active":"2024-04-25T01:08:29.826899+0000","last_peered":"2024-04-25T01:08:29.826899+0000","last_clean":"2024-04-25T01:08:29.826899+0000","last_became_active":"2024-04-25T01:08:28.541136+0000","last_became_peered":"2024-04-25T01:08:28.541136+0000","last_unstale":"2024-04-25T01:08:29.826899+0000","last_undegraded":"2024-04-25T01:08:29.826899+0000","last_fullsized":"2024-04-25T01:08:29.826899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.705353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824739+0000","last_change":"2024-04-25T01:08:24.468504+0000","last_active":"2024-04-25T01:08:29.824739+0000","last_peered":"2024-04-25T01:08:29.824739+0000","last_clean":"2024-04-25T01:08:29.824739+0000","last_became_active":"2024-04-25T01:08:24.468220+0000","last_became_peered":"2024-04-25T01:08:24.468220+0000","last_unstale":"2024-04-25T01:08:29.824739+0000","last_undegraded":"2024-04-25T01:08:29.824739+0000","last_fullsized":"2024-04-25T01:08:29.824739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825079+0000","last_change":"2024-04-25T01:08:23.448344+0000","last_active":"2024-04-25T01:08:29.825079+0000","last_peered":"2024-04-25T01:08:29.825079+0000","last_clean":"2024-04-25T01:08:29.825079+0000","last_became_active":"2024-04-25T01:08:23.447926+0000","last_became_peered":"2024-04-25T01:08:23.447926+0000","last_unstale":"2024-04-25T01:08:29.825079+0000","last_undegraded":"2024-04-25T01:08:29.825079+0000","last_fullsized":"2024-04-25T01:08:29.825079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830344+0000","last_change":"2024-04-25T01:08:28.547149+0000","last_active":"2024-04-25T01:08:29.830344+0000","last_peered":"2024-04-25T01:08:29.830344+0000","last_clean":"2024-04-25T01:08:29.830344+0000","last_became_active":"2024-04-25T01:08:28.546228+0000","last_became_peered":"2024-04-25T01:08:28.546228+0000","last_unstale":"2024-04-25T01:08:29.830344+0000","last_undegraded":"2024-04-25T01:08:29.830344+0000","last_fullsized":"2024-04-25T01:08:29.830344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:58.235046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825749+0000","last_change":"2024-04-25T01:08:24.458889+0000","last_active":"2024-04-25T01:08:29.825749+0000","last_peered":"2024-04-25T01:08:29.825749+0000","last_clean":"2024-04-25T01:08:29.825749+0000","last_became_active":"2024-04-25T01:08:24.458591+0000","last_became_peered":"2024-04-25T01:08:24.458591+0000","last_unstale":"2024-04-25T01:08:29.825749+0000","last_undegraded":"2024-04-25T01:08:29.825749+0000","last_fullsized":"2024-04-25T01:08:29.825749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:14:40.997594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829116+0000","last_change":"2024-04-25T01:08:23.481327+0000","last_active":"2024-04-25T01:08:29.829116+0000","last_peered":"2024-04-25T01:08:29.829116+0000","last_clean":"2024-04-25T01:08:29.829116+0000","last_became_active":"2024-04-25T01:08:23.475537+0000","last_became_peered":"2024-04-25T01:08:23.475537+0000","last_unstale":"2024-04-25T01:08:29.829116+0000","last_undegraded":"2024-04-25T01:08:29.829116+0000","last_fullsized":"2024-04-25T01:08:29.829116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:35.529932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829514+0000","last_change":"2024-04-25T01:08:28.859351+0000","last_active":"2024-04-25T01:08:29.829514+0000","last_peered":"2024-04-25T01:08:29.829514+0000","last_clean":"2024-04-25T01:08:29.829514+0000","last_became_active":"2024-04-25T01:08:28.858099+0000","last_became_peered":"2024-04-25T01:08:28.858099+0000","last_unstale":"2024-04-25T01:08:29.829514+0000","last_undegraded":"2024-04-25T01:08:29.829514+0000","last_fullsized":"2024-04-25T01:08:29.829514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:16:46.276943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508474+0000","last_change":"2024-04-25T01:08:24.495211+0000","last_active":"2024-04-25T01:08:29.508474+0000","last_peered":"2024-04-25T01:08:29.508474+0000","last_clean":"2024-04-25T01:08:29.508474+0000","last_became_active":"2024-04-25T01:08:24.494930+0000","last_became_peered":"2024-04-25T01:08:24.494930+0000","last_unstale":"2024-04-25T01:08:29.508474+0000","last_undegraded":"2024-04-25T01:08:29.508474+0000","last_fullsized":"2024-04-25T01:08:29.508474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825747+0000","last_change":"2024-04-25T01:08:23.461624+0000","last_active":"2024-04-25T01:08:29.825747+0000","last_peered":"2024-04-25T01:08:29.825747+0000","last_clean":"2024-04-25T01:08:29.825747+0000","last_became_active":"2024-04-25T01:08:23.457067+0000","last_became_peered":"2024-04-25T01:08:23.457067+0000","last_unstale":"2024-04-25T01:08:29.825747+0000","last_undegraded":"2024-04-25T01:08:29.825747+0000","last_fullsized":"2024-04-25T01:08:29.825747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829168+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:08:29.829168+0000","last_peered":"2024-04-25T01:08:29.829168+0000","last_clean":"2024-04-25T01:08:29.829168+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:08:29.829168+0000","last_undegraded":"2024-04-25T01:08:29.829168+0000","last_fullsized":"2024-04-25T01:08:29.829168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826455+0000","last_change":"2024-04-25T01:08:28.861776+0000","last_active":"2024-04-25T01:08:29.826455+0000","last_peered":"2024-04-25T01:08:29.826455+0000","last_clean":"2024-04-25T01:08:29.826455+0000","last_became_active":"2024-04-25T01:08:28.860715+0000","last_became_peered":"2024-04-25T01:08:28.860715+0000","last_unstale":"2024-04-25T01:08:29.826455+0000","last_undegraded":"2024-04-25T01:08:29.826455+0000","last_fullsized":"2024-04-25T01:08:29.826455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:54.875040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826011+0000","last_change":"2024-04-25T01:08:24.523631+0000","last_active":"2024-04-25T01:08:29.826011+0000","last_peered":"2024-04-25T01:08:29.826011+0000","last_clean":"2024-04-25T01:08:29.826011+0000","last_became_active":"2024-04-25T01:08:24.523130+0000","last_became_peered":"2024-04-25T01:08:24.523130+0000","last_unstale":"2024-04-25T01:08:29.826011+0000","last_undegraded":"2024-04-25T01:08:29.826011+0000","last_fullsized":"2024-04-25T01:08:29.826011+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:19.861291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826698+0000","last_change":"2024-04-25T01:08:23.470284+0000","last_active":"2024-04-25T01:08:29.826698+0000","last_peered":"2024-04-25T01:08:29.826698+0000","last_clean":"2024-04-25T01:08:29.826698+0000","last_became_active":"2024-04-25T01:08:23.470043+0000","last_became_peered":"2024-04-25T01:08:23.470043+0000","last_unstale":"2024-04-25T01:08:29.826698+0000","last_undegraded":"2024-04-25T01:08:29.826698+0000","last_fullsized":"2024-04-25T01:08:29.826698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827347+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827347+0000","last_peered":"2024-04-25T01:08:29.827347+0000","last_clean":"2024-04-25T01:08:29.827347+0000","last_became_active":"2024-04-25T01:08:28.858545+0000","last_became_peered":"2024-04-25T01:08:28.858545+0000","last_unstale":"2024-04-25T01:08:29.827347+0000","last_undegraded":"2024-04-25T01:08:29.827347+0000","last_fullsized":"2024-04-25T01:08:29.827347+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:57.809358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826582+0000","last_change":"2024-04-25T01:08:24.490983+0000","last_active":"2024-04-25T01:08:29.826582+0000","last_peered":"2024-04-25T01:08:29.826582+0000","last_clean":"2024-04-25T01:08:29.826582+0000","last_became_active":"2024-04-25T01:08:24.490276+0000","last_became_peered":"2024-04-25T01:08:24.490276+0000","last_unstale":"2024-04-25T01:08:29.826582+0000","last_undegraded":"2024-04-25T01:08:29.826582+0000","last_fullsized":"2024-04-25T01:08:29.826582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:17:17.470136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829394+0000","last_change":"2024-04-25T01:08:23.459773+0000","last_active":"2024-04-25T01:08:29.829394+0000","last_peered":"2024-04-25T01:08:29.829394+0000","last_clean":"2024-04-25T01:08:29.829394+0000","last_became_active":"2024-04-25T01:08:23.459374+0000","last_became_peered":"2024-04-25T01:08:23.459374+0000","last_unstale":"2024-04-25T01:08:29.829394+0000","last_undegraded":"2024-04-25T01:08:29.829394+0000","last_fullsized":"2024-04-25T01:08:29.829394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827646+0000","last_change":"2024-04-25T01:08:28.542952+0000","last_active":"2024-04-25T01:08:29.827646+0000","last_peered":"2024-04-25T01:08:29.827646+0000","last_clean":"2024-04-25T01:08:29.827646+0000","last_became_active":"2024-04-25T01:08:28.542388+0000","last_became_peered":"2024-04-25T01:08:28.542388+0000","last_unstale":"2024-04-25T01:08:29.827646+0000","last_undegraded":"2024-04-25T01:08:29.827646+0000","last_fullsized":"2024-04-25T01:08:29.827646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:47.910183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828070+0000","last_change":"2024-04-25T01:08:24.525832+0000","last_active":"2024-04-25T01:08:29.828070+0000","last_peered":"2024-04-25T01:08:29.828070+0000","last_clean":"2024-04-25T01:08:29.828070+0000","last_became_active":"2024-04-25T01:08:24.524963+0000","last_became_peered":"2024-04-25T01:08:24.524963+0000","last_unstale":"2024-04-25T01:08:29.828070+0000","last_undegraded":"2024-04-25T01:08:29.828070+0000","last_fullsized":"2024-04-25T01:08:29.828070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:54.172408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828701+0000","last_change":"2024-04-25T01:08:23.448056+0000","last_active":"2024-04-25T01:08:29.828701+0000","last_peered":"2024-04-25T01:08:29.828701+0000","last_clean":"2024-04-25T01:08:29.828701+0000","last_became_active":"2024-04-25T01:08:23.447748+0000","last_became_peered":"2024-04-25T01:08:23.447748+0000","last_unstale":"2024-04-25T01:08:29.828701+0000","last_undegraded":"2024-04-25T01:08:29.828701+0000","last_fullsized":"2024-04-25T01:08:29.828701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824750+0000","last_change":"2024-04-25T01:08:28.860380+0000","last_active":"2024-04-25T01:08:29.824750+0000","last_peered":"2024-04-25T01:08:29.824750+0000","last_clean":"2024-04-25T01:08:29.824750+0000","last_became_active":"2024-04-25T01:08:28.859396+0000","last_became_peered":"2024-04-25T01:08:28.859396+0000","last_unstale":"2024-04-25T01:08:29.824750+0000","last_undegraded":"2024-04-25T01:08:29.824750+0000","last_fullsized":"2024-04-25T01:08:29.824750+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:28.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826570+0000","last_change":"2024-04-25T01:08:24.524589+0000","last_active":"2024-04-25T01:08:29.826570+0000","last_peered":"2024-04-25T01:08:29.826570+0000","last_clean":"2024-04-25T01:08:29.826570+0000","last_became_active":"2024-04-25T01:08:24.523861+0000","last_became_peered":"2024-04-25T01:08:24.523861+0000","last_unstale":"2024-04-25T01:08:29.826570+0000","last_undegraded":"2024-04-25T01:08:29.826570+0000","last_fullsized":"2024-04-25T01:08:29.826570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:53.746720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.512670+0000","last_change":"2024-04-25T01:08:23.445365+0000","last_active":"2024-04-25T01:08:29.512670+0000","last_peered":"2024-04-25T01:08:29.512670+0000","last_clean":"2024-04-25T01:08:29.512670+0000","last_became_active":"2024-04-25T01:08:23.444505+0000","last_became_peered":"2024-04-25T01:08:23.444505+0000","last_unstale":"2024-04-25T01:08:29.512670+0000","last_undegraded":"2024-04-25T01:08:29.512670+0000","last_fullsized":"2024-04-25T01:08:29.512670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:32:51.459603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510009+0000","last_change":"2024-04-25T01:08:28.549416+0000","last_active":"2024-04-25T01:08:29.510009+0000","last_peered":"2024-04-25T01:08:29.510009+0000","last_clean":"2024-04-25T01:08:29.510009+0000","last_became_active":"2024-04-25T01:08:28.547126+0000","last_became_peered":"2024-04-25T01:08:28.547126+0000","last_unstale":"2024-04-25T01:08:29.510009+0000","last_undegraded":"2024-04-25T01:08:29.510009+0000","last_fullsized":"2024-04-25T01:08:29.510009+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:29.916469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828474+0000","last_change":"2024-04-25T01:08:24.514329+0000","last_active":"2024-04-25T01:08:29.828474+0000","last_peered":"2024-04-25T01:08:29.828474+0000","last_clean":"2024-04-25T01:08:29.828474+0000","last_became_active":"2024-04-25T01:08:24.513670+0000","last_became_peered":"2024-04-25T01:08:24.513670+0000","last_unstale":"2024-04-25T01:08:29.828474+0000","last_undegraded":"2024-04-25T01:08:29.828474+0000","last_fullsized":"2024-04-25T01:08:29.828474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:59.595095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.823985+0000","last_change":"2024-04-25T01:08:23.449156+0000","last_active":"2024-04-25T01:08:29.823985+0000","last_peered":"2024-04-25T01:08:29.823985+0000","last_clean":"2024-04-25T01:08:29.823985+0000","last_became_active":"2024-04-25T01:08:23.448724+0000","last_became_peered":"2024-04-25T01:08:23.448724+0000","last_unstale":"2024-04-25T01:08:29.823985+0000","last_undegraded":"2024-04-25T01:08:29.823985+0000","last_fullsized":"2024-04-25T01:08:29.823985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829295+0000","last_change":"2024-04-25T01:08:28.538540+0000","last_active":"2024-04-25T01:08:29.829295+0000","last_peered":"2024-04-25T01:08:29.829295+0000","last_clean":"2024-04-25T01:08:29.829295+0000","last_became_active":"2024-04-25T01:08:28.537662+0000","last_became_peered":"2024-04-25T01:08:28.537662+0000","last_unstale":"2024-04-25T01:08:29.829295+0000","last_undegraded":"2024-04-25T01:08:29.829295+0000","last_fullsized":"2024-04-25T01:08:29.829295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:06.032667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824689+0000","last_change":"2024-04-25T01:08:24.499087+0000","last_active":"2024-04-25T01:08:29.824689+0000","last_peered":"2024-04-25T01:08:29.824689+0000","last_clean":"2024-04-25T01:08:29.824689+0000","last_became_active":"2024-04-25T01:08:24.498181+0000","last_became_peered":"2024-04-25T01:08:24.498181+0000","last_unstale":"2024-04-25T01:08:29.824689+0000","last_undegraded":"2024-04-25T01:08:29.824689+0000","last_fullsized":"2024-04-25T01:08:29.824689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825380+0000","last_change":"2024-04-25T01:08:23.466687+0000","last_active":"2024-04-25T01:08:29.825380+0000","last_peered":"2024-04-25T01:08:29.825380+0000","last_clean":"2024-04-25T01:08:29.825380+0000","last_became_active":"2024-04-25T01:08:23.465922+0000","last_became_peered":"2024-04-25T01:08:23.465922+0000","last_unstale":"2024-04-25T01:08:29.825380+0000","last_undegraded":"2024-04-25T01:08:29.825380+0000","last_fullsized":"2024-04-25T01:08:29.825380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:54:06.451505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510286+0000","last_change":"2024-04-25T01:08:28.861374+0000","last_active":"2024-04-25T01:08:29.510286+0000","last_peered":"2024-04-25T01:08:29.510286+0000","last_clean":"2024-04-25T01:08:29.510286+0000","last_became_active":"2024-04-25T01:08:28.857471+0000","last_became_peered":"2024-04-25T01:08:28.857471+0000","last_unstale":"2024-04-25T01:08:29.510286+0000","last_undegraded":"2024-04-25T01:08:29.510286+0000","last_fullsized":"2024-04-25T01:08:29.510286+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:58:50.580633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827859+0000","last_change":"2024-04-25T01:08:24.524773+0000","last_active":"2024-04-25T01:08:29.827859+0000","last_peered":"2024-04-25T01:08:29.827859+0000","last_clean":"2024-04-25T01:08:29.827859+0000","last_became_active":"2024-04-25T01:08:24.523273+0000","last_became_peered":"2024-04-25T01:08:24.523273+0000","last_unstale":"2024-04-25T01:08:29.827859+0000","last_undegraded":"2024-04-25T01:08:29.827859+0000","last_fullsized":"2024-04-25T01:08:29.827859+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:53.559257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826090+0000","last_change":"2024-04-25T01:08:23.478405+0000","last_active":"2024-04-25T01:08:29.826090+0000","last_peered":"2024-04-25T01:08:29.826090+0000","last_clean":"2024-04-25T01:08:29.826090+0000","last_became_active":"2024-04-25T01:08:23.477864+0000","last_became_peered":"2024-04-25T01:08:23.477864+0000","last_unstale":"2024-04-25T01:08:29.826090+0000","last_undegraded":"2024-04-25T01:08:29.826090+0000","last_fullsized":"2024-04-25T01:08:29.826090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:14.206151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509288+0000","last_change":"2024-04-25T01:08:28.859485+0000","last_active":"2024-04-25T01:08:29.509288+0000","last_peered":"2024-04-25T01:08:29.509288+0000","last_clean":"2024-04-25T01:08:29.509288+0000","last_became_active":"2024-04-25T01:08:28.858350+0000","last_became_peered":"2024-04-25T01:08:28.858350+0000","last_unstale":"2024-04-25T01:08:29.509288+0000","last_undegraded":"2024-04-25T01:08:29.509288+0000","last_fullsized":"2024-04-25T01:08:29.509288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:27.377063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827382+0000","last_change":"2024-04-25T01:08:24.502358+0000","last_active":"2024-04-25T01:08:29.827382+0000","last_peered":"2024-04-25T01:08:29.827382+0000","last_clean":"2024-04-25T01:08:29.827382+0000","last_became_active":"2024-04-25T01:08:24.502144+0000","last_became_peered":"2024-04-25T01:08:24.502144+0000","last_unstale":"2024-04-25T01:08:29.827382+0000","last_undegraded":"2024-04-25T01:08:29.827382+0000","last_fullsized":"2024-04-25T01:08:29.827382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:17.614542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827872+0000","last_change":"2024-04-25T01:08:23.444558+0000","last_active":"2024-04-25T01:08:29.827872+0000","last_peered":"2024-04-25T01:08:29.827872+0000","last_clean":"2024-04-25T01:08:29.827872+0000","last_became_active":"2024-04-25T01:08:23.444246+0000","last_became_peered":"2024-04-25T01:08:23.444246+0000","last_unstale":"2024-04-25T01:08:29.827872+0000","last_undegraded":"2024-04-25T01:08:29.827872+0000","last_fullsized":"2024-04-25T01:08:29.827872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:44.952485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827499+0000","last_change":"2024-04-25T01:08:28.859590+0000","last_active":"2024-04-25T01:08:29.827499+0000","last_peered":"2024-04-25T01:08:29.827499+0000","last_clean":"2024-04-25T01:08:29.827499+0000","last_became_active":"2024-04-25T01:08:28.858324+0000","last_became_peered":"2024-04-25T01:08:28.858324+0000","last_unstale":"2024-04-25T01:08:29.827499+0000","last_undegraded":"2024-04-25T01:08:29.827499+0000","last_fullsized":"2024-04-25T01:08:29.827499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:49.074275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825751+0000","last_change":"2024-04-25T01:08:24.520516+0000","last_active":"2024-04-25T01:08:29.825751+0000","last_peered":"2024-04-25T01:08:29.825751+0000","last_clean":"2024-04-25T01:08:29.825751+0000","last_became_active":"2024-04-25T01:08:24.519967+0000","last_became_peered":"2024-04-25T01:08:24.519967+0000","last_unstale":"2024-04-25T01:08:29.825751+0000","last_undegraded":"2024-04-25T01:08:29.825751+0000","last_fullsized":"2024-04-25T01:08:29.825751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:50.176772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:35.049426+0000","last_change":"2024-04-25T01:08:23.460787+0000","last_active":"2024-04-25T01:08:35.049426+0000","last_peered":"2024-04-25T01:08:35.049426+0000","last_clean":"2024-04-25T01:08:35.049426+0000","last_became_active":"2024-04-25T01:08:23.455085+0000","last_became_peered":"2024-04-25T01:08:23.455085+0000","last_unstale":"2024-04-25T01:08:35.049426+0000","last_undegraded":"2024-04-25T01:08:35.049426+0000","last_fullsized":"2024-04-25T01:08:35.049426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:26.228957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825396+0000","last_change":"2024-04-25T01:08:28.536251+0000","last_active":"2024-04-25T01:08:29.825396+0000","last_peered":"2024-04-25T01:08:29.825396+0000","last_clean":"2024-04-25T01:08:29.825396+0000","last_became_active":"2024-04-25T01:08:28.535359+0000","last_became_peered":"2024-04-25T01:08:28.535359+0000","last_unstale":"2024-04-25T01:08:29.825396+0000","last_undegraded":"2024-04-25T01:08:29.825396+0000","last_fullsized":"2024-04-25T01:08:29.825396+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:23.923929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511615+0000","last_change":"2024-04-25T01:08:24.505163+0000","last_active":"2024-04-25T01:08:29.511615+0000","last_peered":"2024-04-25T01:08:29.511615+0000","last_clean":"2024-04-25T01:08:29.511615+0000","last_became_active":"2024-04-25T01:08:24.504390+0000","last_became_peered":"2024-04-25T01:08:24.504390+0000","last_unstale":"2024-04-25T01:08:29.511615+0000","last_undegraded":"2024-04-25T01:08:29.511615+0000","last_fullsized":"2024-04-25T01:08:29.511615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829299+0000","last_change":"2024-04-25T01:08:23.484553+0000","last_active":"2024-04-25T01:08:29.829299+0000","last_peered":"2024-04-25T01:08:29.829299+0000","last_clean":"2024-04-25T01:08:29.829299+0000","last_became_active":"2024-04-25T01:08:23.484289+0000","last_became_peered":"2024-04-25T01:08:23.484289+0000","last_unstale":"2024-04-25T01:08:29.829299+0000","last_undegraded":"2024-04-25T01:08:29.829299+0000","last_fullsized":"2024-04-25T01:08:29.829299+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:14.933655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511206+0000","last_change":"2024-04-25T01:08:28.538547+0000","last_active":"2024-04-25T01:08:29.511206+0000","last_peered":"2024-04-25T01:08:29.511206+0000","last_clean":"2024-04-25T01:08:29.511206+0000","last_became_active":"2024-04-25T01:08:28.537497+0000","last_became_peered":"2024-04-25T01:08:28.537497+0000","last_unstale":"2024-04-25T01:08:29.511206+0000","last_undegraded":"2024-04-25T01:08:29.511206+0000","last_fullsized":"2024-04-25T01:08:29.511206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.830135+0000","last_change":"2024-04-25T01:08:24.493527+0000","last_active":"2024-04-25T01:08:29.830135+0000","last_peered":"2024-04-25T01:08:29.830135+0000","last_clean":"2024-04-25T01:08:29.830135+0000","last_became_active":"2024-04-25T01:08:24.493206+0000","last_became_peered":"2024-04-25T01:08:24.493206+0000","last_unstale":"2024-04-25T01:08:29.830135+0000","last_undegraded":"2024-04-25T01:08:29.830135+0000","last_fullsized":"2024-04-25T01:08:29.830135+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826688+0000","last_change":"2024-04-25T01:08:23.476900+0000","last_active":"2024-04-25T01:08:29.826688+0000","last_peered":"2024-04-25T01:08:29.826688+0000","last_clean":"2024-04-25T01:08:29.826688+0000","last_became_active":"2024-04-25T01:08:23.474893+0000","last_became_peered":"2024-04-25T01:08:23.474893+0000","last_unstale":"2024-04-25T01:08:29.826688+0000","last_undegraded":"2024-04-25T01:08:29.826688+0000","last_fullsized":"2024-04-25T01:08:29.826688+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:29:25.208026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508202+0000","last_change":"2024-04-25T01:08:28.545577+0000","last_active":"2024-04-25T01:08:29.508202+0000","last_peered":"2024-04-25T01:08:29.508202+0000","last_clean":"2024-04-25T01:08:29.508202+0000","last_became_active":"2024-04-25T01:08:28.540902+0000","last_became_peered":"2024-04-25T01:08:28.540902+0000","last_unstale":"2024-04-25T01:08:29.508202+0000","last_undegraded":"2024-04-25T01:08:29.508202+0000","last_fullsized":"2024-04-25T01:08:29.508202+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:14.257846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826495+0000","last_change":"2024-04-25T01:08:24.515829+0000","last_active":"2024-04-25T01:08:29.826495+0000","last_peered":"2024-04-25T01:08:29.826495+0000","last_clean":"2024-04-25T01:08:29.826495+0000","last_became_active":"2024-04-25T01:08:24.515267+0000","last_became_peered":"2024-04-25T01:08:24.515267+0000","last_unstale":"2024-04-25T01:08:29.826495+0000","last_undegraded":"2024-04-25T01:08:29.826495+0000","last_fullsized":"2024-04-25T01:08:29.826495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:51.870029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829322+0000","last_change":"2024-04-25T01:08:23.478857+0000","last_active":"2024-04-25T01:08:29.829322+0000","last_peered":"2024-04-25T01:08:29.829322+0000","last_clean":"2024-04-25T01:08:29.829322+0000","last_became_active":"2024-04-25T01:08:23.478215+0000","last_became_peered":"2024-04-25T01:08:23.478215+0000","last_unstale":"2024-04-25T01:08:29.829322+0000","last_undegraded":"2024-04-25T01:08:29.829322+0000","last_fullsized":"2024-04-25T01:08:29.829322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:03:36.304692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508255+0000","last_change":"2024-04-25T01:08:28.863219+0000","last_active":"2024-04-25T01:08:29.508255+0000","last_peered":"2024-04-25T01:08:29.508255+0000","last_clean":"2024-04-25T01:08:29.508255+0000","last_became_active":"2024-04-25T01:08:28.862202+0000","last_became_peered":"2024-04-25T01:08:28.862202+0000","last_unstale":"2024-04-25T01:08:29.508255+0000","last_undegraded":"2024-04-25T01:08:29.508255+0000","last_fullsized":"2024-04-25T01:08:29.508255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:52:58.057652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829042+0000","last_change":"2024-04-25T01:08:28.864029+0000","last_active":"2024-04-25T01:08:29.829042+0000","last_peered":"2024-04-25T01:08:29.829042+0000","last_clean":"2024-04-25T01:08:29.829042+0000","last_became_active":"2024-04-25T01:08:28.863040+0000","last_became_peered":"2024-04-25T01:08:28.863040+0000","last_unstale":"2024-04-25T01:08:29.829042+0000","last_undegraded":"2024-04-25T01:08:29.829042+0000","last_fullsized":"2024-04-25T01:08:29.829042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:55.932667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827904+0000","last_change":"2024-04-25T01:08:24.472094+0000","last_active":"2024-04-25T01:08:29.827904+0000","last_peered":"2024-04-25T01:08:29.827904+0000","last_clean":"2024-04-25T01:08:29.827904+0000","last_became_active":"2024-04-25T01:08:24.471381+0000","last_became_peered":"2024-04-25T01:08:24.471381+0000","last_unstale":"2024-04-25T01:08:29.827904+0000","last_undegraded":"2024-04-25T01:08:29.827904+0000","last_fullsized":"2024-04-25T01:08:29.827904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825540+0000","last_change":"2024-04-25T01:08:23.470504+0000","last_active":"2024-04-25T01:08:29.825540+0000","last_peered":"2024-04-25T01:08:29.825540+0000","last_clean":"2024-04-25T01:08:29.825540+0000","last_became_active":"2024-04-25T01:08:23.470060+0000","last_became_peered":"2024-04-25T01:08:23.470060+0000","last_unstale":"2024-04-25T01:08:29.825540+0000","last_undegraded":"2024-04-25T01:08:29.825540+0000","last_fullsized":"2024-04-25T01:08:29.825540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:52.050718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829192+0000","last_change":"2024-04-25T01:08:28.544590+0000","last_active":"2024-04-25T01:08:29.829192+0000","last_peered":"2024-04-25T01:08:29.829192+0000","last_clean":"2024-04-25T01:08:29.829192+0000","last_became_active":"2024-04-25T01:08:28.543195+0000","last_became_peered":"2024-04-25T01:08:28.543195+0000","last_unstale":"2024-04-25T01:08:29.829192+0000","last_undegraded":"2024-04-25T01:08:29.829192+0000","last_fullsized":"2024-04-25T01:08:29.829192+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:57.621895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508183+0000","last_change":"2024-04-25T01:08:24.494258+0000","last_active":"2024-04-25T01:08:29.508183+0000","last_peered":"2024-04-25T01:08:29.508183+0000","last_clean":"2024-04-25T01:08:29.508183+0000","last_became_active":"2024-04-25T01:08:24.493935+0000","last_became_peered":"2024-04-25T01:08:24.493935+0000","last_unstale":"2024-04-25T01:08:29.508183+0000","last_undegraded":"2024-04-25T01:08:29.508183+0000","last_fullsized":"2024-04-25T01:08:29.508183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508536+0000","last_change":"2024-04-25T01:08:23.463618+0000","last_active":"2024-04-25T01:08:29.508536+0000","last_peered":"2024-04-25T01:08:29.508536+0000","last_clean":"2024-04-25T01:08:29.508536+0000","last_became_active":"2024-04-25T01:08:23.463303+0000","last_became_peered":"2024-04-25T01:08:23.463303+0000","last_unstale":"2024-04-25T01:08:29.508536+0000","last_undegraded":"2024-04-25T01:08:29.508536+0000","last_fullsized":"2024-04-25T01:08:29.508536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:36.944771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828071+0000","last_change":"2024-04-25T01:08:23.465959+0000","last_active":"2024-04-25T01:08:29.828071+0000","last_peered":"2024-04-25T01:08:29.828071+0000","last_clean":"2024-04-25T01:08:29.828071+0000","last_became_active":"2024-04-25T01:08:23.465518+0000","last_became_peered":"2024-04-25T01:08:23.465518+0000","last_unstale":"2024-04-25T01:08:29.828071+0000","last_undegraded":"2024-04-25T01:08:29.828071+0000","last_fullsized":"2024-04-25T01:08:29.828071+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:38.433474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827782+0000","last_change":"2024-04-25T01:08:24.493189+0000","last_active":"2024-04-25T01:08:29.827782+0000","last_peered":"2024-04-25T01:08:29.827782+0000","last_clean":"2024-04-25T01:08:29.827782+0000","last_became_active":"2024-04-25T01:08:24.473740+0000","last_became_peered":"2024-04-25T01:08:24.473740+0000","last_unstale":"2024-04-25T01:08:29.827782+0000","last_undegraded":"2024-04-25T01:08:29.827782+0000","last_fullsized":"2024-04-25T01:08:29.827782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825925+0000","last_change":"2024-04-25T01:08:28.542530+0000","last_active":"2024-04-25T01:08:29.825925+0000","last_peered":"2024-04-25T01:08:29.825925+0000","last_clean":"2024-04-25T01:08:29.825925+0000","last_became_active":"2024-04-25T01:08:28.541445+0000","last_became_peered":"2024-04-25T01:08:28.541445+0000","last_unstale":"2024-04-25T01:08:29.825925+0000","last_undegraded":"2024-04-25T01:08:29.825925+0000","last_fullsized":"2024-04-25T01:08:29.825925+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:31:21.963089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508697+0000","last_change":"2024-04-25T01:08:23.475915+0000","last_active":"2024-04-25T01:08:29.508697+0000","last_peered":"2024-04-25T01:08:29.508697+0000","last_clean":"2024-04-25T01:08:29.508697+0000","last_became_active":"2024-04-25T01:08:23.475628+0000","last_became_peered":"2024-04-25T01:08:23.475628+0000","last_unstale":"2024-04-25T01:08:29.508697+0000","last_undegraded":"2024-04-25T01:08:29.508697+0000","last_fullsized":"2024-04-25T01:08:29.508697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:22.410694+0000","last_clean_scrub_stamp":"2024-04-25T01:08:22.410694+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:56.721831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511101+0000","last_change":"2024-04-25T01:08:24.476673+0000","last_active":"2024-04-25T01:08:29.511101+0000","last_peered":"2024-04-25T01:08:29.511101+0000","last_clean":"2024-04-25T01:08:29.511101+0000","last_became_active":"2024-04-25T01:08:24.471388+0000","last_became_peered":"2024-04-25T01:08:24.471388+0000","last_unstale":"2024-04-25T01:08:29.511101+0000","last_undegraded":"2024-04-25T01:08:29.511101+0000","last_fullsized":"2024-04-25T01:08:29.511101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:32:36.539667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509254+0000","last_change":"2024-04-25T01:08:28.539720+0000","last_active":"2024-04-25T01:08:29.509254+0000","last_peered":"2024-04-25T01:08:29.509254+0000","last_clean":"2024-04-25T01:08:29.509254+0000","last_became_active":"2024-04-25T01:08:28.538885+0000","last_became_peered":"2024-04-25T01:08:28.538885+0000","last_unstale":"2024-04-25T01:08:29.509254+0000","last_undegraded":"2024-04-25T01:08:29.509254+0000","last_fullsized":"2024-04-25T01:08:29.509254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:27.483067+0000","last_clean_scrub_stamp":"2024-04-25T01:08:27.483067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:39:51.273909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824379+0000","last_change":"2024-04-25T01:08:24.499102+0000","last_active":"2024-04-25T01:08:29.824379+0000","last_peered":"2024-04-25T01:08:29.824379+0000","last_clean":"2024-04-25T01:08:29.824379+0000","last_became_active":"2024-04-25T01:08:24.498282+0000","last_became_peered":"2024-04-25T01:08:24.498282+0000","last_unstale":"2024-04-25T01:08:29.824379+0000","last_undegraded":"2024-04-25T01:08:29.824379+0000","last_fullsized":"2024-04-25T01:08:29.824379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.824785+0000","last_change":"2024-04-25T01:08:24.477024+0000","last_active":"2024-04-25T01:08:29.824785+0000","last_peered":"2024-04-25T01:08:29.824785+0000","last_clean":"2024-04-25T01:08:29.824785+0000","last_became_active":"2024-04-25T01:08:24.476723+0000","last_became_peered":"2024-04-25T01:08:24.476723+0000","last_unstale":"2024-04-25T01:08:29.824785+0000","last_undegraded":"2024-04-25T01:08:29.824785+0000","last_fullsized":"2024-04-25T01:08:29.824785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825769+0000","last_change":"2024-04-25T01:08:24.492781+0000","last_active":"2024-04-25T01:08:29.825769+0000","last_peered":"2024-04-25T01:08:29.825769+0000","last_clean":"2024-04-25T01:08:29.825769+0000","last_became_active":"2024-04-25T01:08:24.492041+0000","last_became_peered":"2024-04-25T01:08:24.492041+0000","last_unstale":"2024-04-25T01:08:29.825769+0000","last_undegraded":"2024-04-25T01:08:29.825769+0000","last_fullsized":"2024-04-25T01:08:29.825769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:11:35.171404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827611+0000","last_change":"2024-04-25T01:08:24.493915+0000","last_active":"2024-04-25T01:08:29.827611+0000","last_peered":"2024-04-25T01:08:29.827611+0000","last_clean":"2024-04-25T01:08:29.827611+0000","last_became_active":"2024-04-25T01:08:24.493620+0000","last_became_peered":"2024-04-25T01:08:24.493620+0000","last_unstale":"2024-04-25T01:08:29.827611+0000","last_undegraded":"2024-04-25T01:08:29.827611+0000","last_fullsized":"2024-04-25T01:08:29.827611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:42.788724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.509565+0000","last_change":"2024-04-25T01:08:24.491720+0000","last_active":"2024-04-25T01:08:29.509565+0000","last_peered":"2024-04-25T01:08:29.509565+0000","last_clean":"2024-04-25T01:08:29.509565+0000","last_became_active":"2024-04-25T01:08:24.491474+0000","last_became_peered":"2024-04-25T01:08:24.491474+0000","last_unstale":"2024-04-25T01:08:29.509565+0000","last_undegraded":"2024-04-25T01:08:29.509565+0000","last_fullsized":"2024-04-25T01:08:29.509565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828668+0000","last_change":"2024-04-25T01:08:24.472108+0000","last_active":"2024-04-25T01:08:29.828668+0000","last_peered":"2024-04-25T01:08:29.828668+0000","last_clean":"2024-04-25T01:08:29.828668+0000","last_became_active":"2024-04-25T01:08:24.471688+0000","last_became_peered":"2024-04-25T01:08:24.471688+0000","last_unstale":"2024-04-25T01:08:29.828668+0000","last_undegraded":"2024-04-25T01:08:29.828668+0000","last_fullsized":"2024-04-25T01:08:29.828668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829144+0000","last_change":"2024-04-25T01:08:24.515413+0000","last_active":"2024-04-25T01:08:29.829144+0000","last_peered":"2024-04-25T01:08:29.829144+0000","last_clean":"2024-04-25T01:08:29.829144+0000","last_became_active":"2024-04-25T01:08:24.514573+0000","last_became_peered":"2024-04-25T01:08:24.514573+0000","last_unstale":"2024-04-25T01:08:29.829144+0000","last_undegraded":"2024-04-25T01:08:29.829144+0000","last_fullsized":"2024-04-25T01:08:29.829144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.510173+0000","last_change":"2024-04-25T01:08:24.487417+0000","last_active":"2024-04-25T01:08:29.510173+0000","last_peered":"2024-04-25T01:08:29.510173+0000","last_clean":"2024-04-25T01:08:29.510173+0000","last_became_active":"2024-04-25T01:08:24.487135+0000","last_became_peered":"2024-04-25T01:08:24.487135+0000","last_unstale":"2024-04-25T01:08:29.510173+0000","last_undegraded":"2024-04-25T01:08:29.510173+0000","last_fullsized":"2024-04-25T01:08:29.510173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825289+0000","last_change":"2024-04-25T01:08:24.484688+0000","last_active":"2024-04-25T01:08:29.825289+0000","last_peered":"2024-04-25T01:08:29.825289+0000","last_clean":"2024-04-25T01:08:29.825289+0000","last_became_active":"2024-04-25T01:08:24.483926+0000","last_became_peered":"2024-04-25T01:08:24.483926+0000","last_unstale":"2024-04-25T01:08:29.825289+0000","last_undegraded":"2024-04-25T01:08:29.825289+0000","last_fullsized":"2024-04-25T01:08:29.825289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:10:55.529308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508350+0000","last_change":"2024-04-25T01:08:24.487479+0000","last_active":"2024-04-25T01:08:29.508350+0000","last_peered":"2024-04-25T01:08:29.508350+0000","last_clean":"2024-04-25T01:08:29.508350+0000","last_became_active":"2024-04-25T01:08:24.487177+0000","last_became_peered":"2024-04-25T01:08:24.487177+0000","last_unstale":"2024-04-25T01:08:29.508350+0000","last_undegraded":"2024-04-25T01:08:29.508350+0000","last_fullsized":"2024-04-25T01:08:29.508350+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:15:15.315379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825549+0000","last_change":"2024-04-25T01:08:24.516077+0000","last_active":"2024-04-25T01:08:29.825549+0000","last_peered":"2024-04-25T01:08:29.825549+0000","last_clean":"2024-04-25T01:08:29.825549+0000","last_became_active":"2024-04-25T01:08:24.515560+0000","last_became_peered":"2024-04-25T01:08:24.515560+0000","last_unstale":"2024-04-25T01:08:29.825549+0000","last_undegraded":"2024-04-25T01:08:29.825549+0000","last_fullsized":"2024-04-25T01:08:29.825549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:28:30.731050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827066+0000","last_change":"2024-04-25T01:08:24.497188+0000","last_active":"2024-04-25T01:08:29.827066+0000","last_peered":"2024-04-25T01:08:29.827066+0000","last_clean":"2024-04-25T01:08:29.827066+0000","last_became_active":"2024-04-25T01:08:24.494236+0000","last_became_peered":"2024-04-25T01:08:24.494236+0000","last_unstale":"2024-04-25T01:08:29.827066+0000","last_undegraded":"2024-04-25T01:08:29.827066+0000","last_fullsized":"2024-04-25T01:08:29.827066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:27:27.907903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825891+0000","last_change":"2024-04-25T01:08:24.497623+0000","last_active":"2024-04-25T01:08:29.825891+0000","last_peered":"2024-04-25T01:08:29.825891+0000","last_clean":"2024-04-25T01:08:29.825891+0000","last_became_active":"2024-04-25T01:08:24.488300+0000","last_became_peered":"2024-04-25T01:08:24.488300+0000","last_unstale":"2024-04-25T01:08:29.825891+0000","last_undegraded":"2024-04-25T01:08:29.825891+0000","last_fullsized":"2024-04-25T01:08:29.825891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:06:50.021919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826575+0000","last_change":"2024-04-25T01:08:24.525066+0000","last_active":"2024-04-25T01:08:29.826575+0000","last_peered":"2024-04-25T01:08:29.826575+0000","last_clean":"2024-04-25T01:08:29.826575+0000","last_became_active":"2024-04-25T01:08:24.524022+0000","last_became_peered":"2024-04-25T01:08:24.524022+0000","last_unstale":"2024-04-25T01:08:29.826575+0000","last_undegraded":"2024-04-25T01:08:29.826575+0000","last_fullsized":"2024-04-25T01:08:29.826575+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:33.317294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828523+0000","last_change":"2024-04-25T01:08:24.474988+0000","last_active":"2024-04-25T01:08:29.828523+0000","last_peered":"2024-04-25T01:08:29.828523+0000","last_clean":"2024-04-25T01:08:29.828523+0000","last_became_active":"2024-04-25T01:08:24.472840+0000","last_became_peered":"2024-04-25T01:08:24.472840+0000","last_unstale":"2024-04-25T01:08:29.828523+0000","last_undegraded":"2024-04-25T01:08:29.828523+0000","last_fullsized":"2024-04-25T01:08:29.828523+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:20:23.105129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827354+0000","last_change":"2024-04-25T01:08:24.504414+0000","last_active":"2024-04-25T01:08:29.827354+0000","last_peered":"2024-04-25T01:08:29.827354+0000","last_clean":"2024-04-25T01:08:29.827354+0000","last_became_active":"2024-04-25T01:08:24.503622+0000","last_became_peered":"2024-04-25T01:08:24.503622+0000","last_unstale":"2024-04-25T01:08:29.827354+0000","last_undegraded":"2024-04-25T01:08:29.827354+0000","last_fullsized":"2024-04-25T01:08:29.827354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:01.970030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825048+0000","last_change":"2024-04-25T01:08:24.496532+0000","last_active":"2024-04-25T01:08:29.825048+0000","last_peered":"2024-04-25T01:08:29.825048+0000","last_clean":"2024-04-25T01:08:29.825048+0000","last_became_active":"2024-04-25T01:08:24.495812+0000","last_became_peered":"2024-04-25T01:08:24.495812+0000","last_unstale":"2024-04-25T01:08:29.825048+0000","last_undegraded":"2024-04-25T01:08:29.825048+0000","last_fullsized":"2024-04-25T01:08:29.825048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:41:57.575027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828165+0000","last_change":"2024-04-25T01:08:24.461285+0000","last_active":"2024-04-25T01:08:29.828165+0000","last_peered":"2024-04-25T01:08:29.828165+0000","last_clean":"2024-04-25T01:08:29.828165+0000","last_became_active":"2024-04-25T01:08:24.460929+0000","last_became_peered":"2024-04-25T01:08:24.460929+0000","last_unstale":"2024-04-25T01:08:29.828165+0000","last_undegraded":"2024-04-25T01:08:29.828165+0000","last_fullsized":"2024-04-25T01:08:29.828165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:08.805746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827126+0000","last_change":"2024-04-25T01:08:24.493618+0000","last_active":"2024-04-25T01:08:29.827126+0000","last_peered":"2024-04-25T01:08:29.827126+0000","last_clean":"2024-04-25T01:08:29.827126+0000","last_became_active":"2024-04-25T01:08:24.489439+0000","last_became_peered":"2024-04-25T01:08:24.489439+0000","last_unstale":"2024-04-25T01:08:29.827126+0000","last_undegraded":"2024-04-25T01:08:29.827126+0000","last_fullsized":"2024-04-25T01:08:29.827126+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:43.940192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.508267+0000","last_change":"2024-04-25T01:08:24.478846+0000","last_active":"2024-04-25T01:08:29.508267+0000","last_peered":"2024-04-25T01:08:29.508267+0000","last_clean":"2024-04-25T01:08:29.508267+0000","last_became_active":"2024-04-25T01:08:24.478580+0000","last_became_peered":"2024-04-25T01:08:24.478580+0000","last_unstale":"2024-04-25T01:08:29.508267+0000","last_undegraded":"2024-04-25T01:08:29.508267+0000","last_fullsized":"2024-04-25T01:08:29.508267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:45.754167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.511438+0000","last_change":"2024-04-25T01:08:24.481980+0000","last_active":"2024-04-25T01:08:29.511438+0000","last_peered":"2024-04-25T01:08:29.511438+0000","last_clean":"2024-04-25T01:08:29.511438+0000","last_became_active":"2024-04-25T01:08:24.480214+0000","last_became_peered":"2024-04-25T01:08:24.480214+0000","last_unstale":"2024-04-25T01:08:29.511438+0000","last_undegraded":"2024-04-25T01:08:29.511438+0000","last_fullsized":"2024-04-25T01:08:29.511438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:14:15.943389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826879+0000","last_change":"2024-04-25T01:08:24.480212+0000","last_active":"2024-04-25T01:08:29.826879+0000","last_peered":"2024-04-25T01:08:29.826879+0000","last_clean":"2024-04-25T01:08:29.826879+0000","last_became_active":"2024-04-25T01:08:24.477781+0000","last_became_peered":"2024-04-25T01:08:24.477781+0000","last_unstale":"2024-04-25T01:08:29.826879+0000","last_undegraded":"2024-04-25T01:08:29.826879+0000","last_fullsized":"2024-04-25T01:08:29.826879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829093+0000","last_change":"2024-04-25T01:08:24.480926+0000","last_active":"2024-04-25T01:08:29.829093+0000","last_peered":"2024-04-25T01:08:29.829093+0000","last_clean":"2024-04-25T01:08:29.829093+0000","last_became_active":"2024-04-25T01:08:24.477787+0000","last_became_peered":"2024-04-25T01:08:24.477787+0000","last_unstale":"2024-04-25T01:08:29.829093+0000","last_undegraded":"2024-04-25T01:08:29.829093+0000","last_fullsized":"2024-04-25T01:08:29.829093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:53.361248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827821+0000","last_change":"2024-04-25T01:08:24.525497+0000","last_active":"2024-04-25T01:08:29.827821+0000","last_peered":"2024-04-25T01:08:29.827821+0000","last_clean":"2024-04-25T01:08:29.827821+0000","last_became_active":"2024-04-25T01:08:24.524512+0000","last_became_peered":"2024-04-25T01:08:24.524512+0000","last_unstale":"2024-04-25T01:08:29.827821+0000","last_undegraded":"2024-04-25T01:08:29.827821+0000","last_fullsized":"2024-04-25T01:08:29.827821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:35.642716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.507922+0000","last_change":"2024-04-25T01:08:24.490653+0000","last_active":"2024-04-25T01:08:29.507922+0000","last_peered":"2024-04-25T01:08:29.507922+0000","last_clean":"2024-04-25T01:08:29.507922+0000","last_became_active":"2024-04-25T01:08:24.486493+0000","last_became_peered":"2024-04-25T01:08:24.486493+0000","last_unstale":"2024-04-25T01:08:29.507922+0000","last_undegraded":"2024-04-25T01:08:29.507922+0000","last_fullsized":"2024-04-25T01:08:29.507922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:35:56.456964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828202+0000","last_change":"2024-04-25T01:08:24.510776+0000","last_active":"2024-04-25T01:08:29.828202+0000","last_peered":"2024-04-25T01:08:29.828202+0000","last_clean":"2024-04-25T01:08:29.828202+0000","last_became_active":"2024-04-25T01:08:24.509980+0000","last_became_peered":"2024-04-25T01:08:24.509980+0000","last_unstale":"2024-04-25T01:08:29.828202+0000","last_undegraded":"2024-04-25T01:08:29.828202+0000","last_fullsized":"2024-04-25T01:08:29.828202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:54.603109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.825321+0000","last_change":"2024-04-25T01:08:24.497109+0000","last_active":"2024-04-25T01:08:29.825321+0000","last_peered":"2024-04-25T01:08:29.825321+0000","last_clean":"2024-04-25T01:08:29.825321+0000","last_became_active":"2024-04-25T01:08:24.496213+0000","last_became_peered":"2024-04-25T01:08:24.496213+0000","last_unstale":"2024-04-25T01:08:29.825321+0000","last_undegraded":"2024-04-25T01:08:29.825321+0000","last_fullsized":"2024-04-25T01:08:29.825321+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:18.447082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827021+0000","last_change":"2024-04-25T01:08:24.524338+0000","last_active":"2024-04-25T01:08:29.827021+0000","last_peered":"2024-04-25T01:08:29.827021+0000","last_clean":"2024-04-25T01:08:29.827021+0000","last_became_active":"2024-04-25T01:08:24.523936+0000","last_became_peered":"2024-04-25T01:08:24.523936+0000","last_unstale":"2024-04-25T01:08:29.827021+0000","last_undegraded":"2024-04-25T01:08:29.827021+0000","last_fullsized":"2024-04-25T01:08:29.827021+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:37:50.812402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.826302+0000","last_change":"2024-04-25T01:08:24.491543+0000","last_active":"2024-04-25T01:08:29.826302+0000","last_peered":"2024-04-25T01:08:29.826302+0000","last_clean":"2024-04-25T01:08:29.826302+0000","last_became_active":"2024-04-25T01:08:24.490866+0000","last_became_peered":"2024-04-25T01:08:24.490866+0000","last_unstale":"2024-04-25T01:08:29.826302+0000","last_undegraded":"2024-04-25T01:08:29.826302+0000","last_fullsized":"2024-04-25T01:08:29.826302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:38.093629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.828785+0000","last_change":"2024-04-25T01:08:24.459335+0000","last_active":"2024-04-25T01:08:29.828785+0000","last_peered":"2024-04-25T01:08:29.828785+0000","last_clean":"2024-04-25T01:08:29.828785+0000","last_became_active":"2024-04-25T01:08:24.458795+0000","last_became_peered":"2024-04-25T01:08:24.458795+0000","last_unstale":"2024-04-25T01:08:29.828785+0000","last_undegraded":"2024-04-25T01:08:29.828785+0000","last_fullsized":"2024-04-25T01:08:29.828785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:10.682483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.829652+0000","last_change":"2024-04-25T01:08:24.465465+0000","last_active":"2024-04-25T01:08:29.829652+0000","last_peered":"2024-04-25T01:08:29.829652+0000","last_clean":"2024-04-25T01:08:29.829652+0000","last_became_active":"2024-04-25T01:08:24.464433+0000","last_became_peered":"2024-04-25T01:08:24.464433+0000","last_unstale":"2024-04-25T01:08:29.829652+0000","last_undegraded":"2024-04-25T01:08:29.829652+0000","last_fullsized":"2024-04-25T01:08:29.829652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:40.075681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T01:08:29.827333+0000","last_change":"2024-04-25T01:08:24.525387+0000","last_active":"2024-04-25T01:08:29.827333+0000","last_peered":"2024-04-25T01:08:29.827333+0000","last_clean":"2024-04-25T01:08:29.827333+0000","last_became_active":"2024-04-25T01:08:24.524324+0000","last_became_peered":"2024-04-25T01:08:24.524324+0000","last_unstale":"2024-04-25T01:08:29.827333+0000","last_undegraded":"2024-04-25T01:08:29.827333+0000","last_fullsized":"2024-04-25T01:08:29.827333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:23.420429+0000","last_clean_scrub_stamp":"2024-04-25T01:08:23.420429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:58:49.276483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":856064,"data_stored":693413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509450,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":102589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":270336,"data_stored":106305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":258048,"data_stored":102463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":102553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":864256,"data_stored":693469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:08:46.911 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T01:08:46.911 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T01:08:46.911 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T01:08:46.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T01:08:47.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.073+0000 7f03fffba640 1 Processor -- start 2024-04-25T01:08:47.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.073+0000 7f03fffba640 1 -- start start 2024-04-25T01:08:47.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03f80d48d0 0x7f03f80cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f80d10b0 con 0x7f03f80d03e0 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f03f80d1de0 con 0x7f03f80d48d0 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03f80d1390 con 0x7f03f815e470 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03f80d48d0 0x7f03f80cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03f80d48d0 0x7f03f80cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50276/0 (socket says 172.21.15.33:50276) 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 -- 172.21.15.33:0/1896417728 learned_addr learned my addr 172.21.15.33:0/1896417728 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fd555640 1 --2- 172.21.15.33:0/1896417728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 -- 172.21.15.33:0/1896417728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 msgr2=0x7f03f80ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:47.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 --2- 172.21.15.33:0/1896417728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 -- 172.21.15.33:0/1896417728 >> v1:172.21.15.138:6789/0 conn(0x7f03f80d03e0 legacy=0x7f03f80d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 -- 172.21.15.33:0/1896417728 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f80d1670 con 0x7f03f80d48d0 2024-04-25T01:08:47.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fe557640 1 --2- 172.21.15.33:0/1896417728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f80cef00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009e50 tx=0x7f03f001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83333703222de47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fcd54640 1 -- 172.21.15.33:0/1896417728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f00a5770 con 0x7f03f80d48d0 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fcd54640 1 -- 172.21.15.33:0/1896417728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f00a5910 con 0x7f03f80d48d0 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fcd54640 1 -- 172.21.15.33:0/1896417728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f00a5ce0 con 0x7f03f80d48d0 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- 172.21.15.33:0/1896417728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 msgr2=0x7f03f80cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 --2- 172.21.15.33:0/1896417728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f80cef00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009e50 tx=0x7f03f001c9a0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- 172.21.15.33:0/1896417728 shutdown_connections 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 --2- 172.21.15.33:0/1896417728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f80cef00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 --2- 172.21.15.33:0/1896417728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- 172.21.15.33:0/1896417728 >> 172.21.15.33:0/1896417728 conn(0x7f03f80d3db0 msgr2=0x7f03f80d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- 172.21.15.33:0/1896417728 shutdown_connections 2024-04-25T01:08:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.077+0000 7f03fffba640 1 -- 172.21.15.33:0/1896417728 wait complete. 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 Processor -- start 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 -- start start 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f80d03e0 0x7f03f80c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f815fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03f816adc0 con 0x7f03f80d48d0 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03f80cfea0 con 0x7f03f80d03e0 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03f816af40 con 0x7f03f815e470 2024-04-25T01:08:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f815fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f815fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50278/0 (socket says 172.21.15.33:50278) 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 -- 172.21.15.33:0/2298138962 learned_addr learned my addr 172.21.15.33:0/2298138962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fe557640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fdd56640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f80d03e0 0x7f03f80c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 -- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 msgr2=0x7f03f80cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 -- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f80d03e0 msgr2=0x7f03f80c9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f80d03e0 0x7f03f80c9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 -- 172.21.15.33:0/2298138962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f81559b0 con 0x7f03f80d48d0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fd555640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f815fb20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00d620 tx=0x7f03ec00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03ea7fc640 1 -- 172.21.15.33:0/2298138962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00b620 con 0x7f03f80d48d0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03ea7fc640 1 -- 172.21.15.33:0/2298138962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec019070 con 0x7f03f80d48d0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f80cb0d0 con 0x7f03f80d48d0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03ea7fc640 1 -- 172.21.15.33:0/2298138962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00b9e0 con 0x7f03f80d48d0 2024-04-25T01:08:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f80cb690 con 0x7f03f80d48d0 2024-04-25T01:08:47.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f8069040 con 0x7f03f80d48d0 2024-04-25T01:08:47.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.081+0000 7f03ea7fc640 1 -- 172.21.15.33:0/2298138962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f03ec00bb80 con 0x7f03f80d48d0 2024-04-25T01:08:47.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.085+0000 7f03ea7fc640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03dc0678f0 0x7f03dc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.085+0000 7f03ea7fc640 1 -- 172.21.15.33:0/2298138962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f03ec086340 con 0x7f03f80d48d0 2024-04-25T01:08:47.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.085+0000 7f03fdd56640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03dc0678f0 0x7f03dc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.085+0000 7f03fdd56640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03dc0678f0 0x7f03dc069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001650 tx=0x7f03f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.089+0000 7f03ea7fc640 1 -- 172.21.15.33:0/2298138962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f03ec056bd0 con 0x7f03f80d48d0 2024-04-25T01:08:47.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.277+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f03f806ce00 con 0x7f03f80d48d0 2024-04-25T01:08:47.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.277+0000 7f03ea7fc640 1 -- 172.21.15.33:0/2298138962 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f03ec0564d0 con 0x7f03f80d48d0 2024-04-25T01:08:47.271 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:47.271 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03dc0678f0 msgr2=0x7f03dc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03dc0678f0 0x7f03dc069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f03f4001650 tx=0x7f03f4009040 comp rx=0 tx=0).stop 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 msgr2=0x7f03f815fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f815fb20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00d620 tx=0x7f03ec00dae0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 shutdown_connections 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03dc0678f0 0x7f03dc069db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03f815e470 0x7f03f80cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03f80d48d0 0x7f03f815fb20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 --2- 172.21.15.33:0/2298138962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03f80d03e0 0x7f03f80c9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 >> 172.21.15.33:0/2298138962 conn(0x7f03f80d3db0 msgr2=0x7f03f80b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 shutdown_connections 2024-04-25T01:08:47.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.281+0000 7f03fffba640 1 -- 172.21.15.33:0/2298138962 wait complete. 2024-04-25T01:08:47.289 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T01:08:47.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:47.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e4d41d640 1 Processor -- start 2024-04-25T01:08:47.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e4d41d640 1 -- start start 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e4d41d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e480d03e0 0x7f9e480d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e4d41d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e480d48d0 0x7f9e480cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e4d41d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e480d10b0 con 0x7f9e4815e470 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e4d41d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f9e480cff10 con 0x7f9e480d48d0 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e4d41d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9e480d1de0 con 0x7f9e480d03e0 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e477fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9e480d48d0 0x7f9e480cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e467fc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9e4815e470 0x7f9e480d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34006/0 (socket says 172.21.15.33:34006) 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e467fc640 1 -- 172.21.15.33:0/1320457783 learned_addr learned my addr 172.21.15.33:0/1320457783 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e46ffd640 1 --2- 172.21.15.33:0/1320457783 >> v2:172.21.15.138:3300/0 conn(0x7f9e480d03e0 0x7f9e480d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e45ffb640 1 -- 172.21.15.33:0/1320457783 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1096378933 0 0) 0x7f9e480d10b0 con 0x7f9e4815e470 2024-04-25T01:08:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e45ffb640 1 -- 172.21.15.33:0/1320457783 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e30003530 con 0x7f9e4815e470 2024-04-25T01:08:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e477fe640 1 -- 172.21.15.33:0/1320457783 >> v2:172.21.15.138:3300/0 conn(0x7f9e480d03e0 msgr2=0x7f9e480d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e477fe640 1 --2- 172.21.15.33:0/1320457783 >> v2:172.21.15.138:3300/0 conn(0x7f9e480d03e0 0x7f9e480d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e477fe640 1 -- 172.21.15.33:0/1320457783 >> v1:172.21.15.33:6789/0 conn(0x7f9e4815e470 legacy=0x7f9e480d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e477fe640 1 -- 172.21.15.33:0/1320457783 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e480d1390 con 0x7f9e480d48d0 2024-04-25T01:08:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.417+0000 7f9e477fe640 1 --2- 172.21.15.33:0/1320457783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d48d0 0x7f9e480cf9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9e4808c7a0 tx=0x7f9e4001c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ff6a8ef75ec650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e45ffb640 1 -- 172.21.15.33:0/1320457783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e400b2070 con 0x7f9e480d48d0 2024-04-25T01:08:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e45ffb640 1 -- 172.21.15.33:0/1320457783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e400a5860 con 0x7f9e480d48d0 2024-04-25T01:08:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e45ffb640 1 -- 172.21.15.33:0/1320457783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e400a5c30 con 0x7f9e480d48d0 2024-04-25T01:08:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- 172.21.15.33:0/1320457783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d48d0 msgr2=0x7f9e480cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/1320457783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d48d0 0x7f9e480cf9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9e4808c7a0 tx=0x7f9e4001c840 comp rx=0 tx=0).stop 2024-04-25T01:08:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- 172.21.15.33:0/1320457783 shutdown_connections 2024-04-25T01:08:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/1320457783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d48d0 0x7f9e480cf9d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/1320457783 >> v2:172.21.15.138:3300/0 conn(0x7f9e480d03e0 0x7f9e480d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- 172.21.15.33:0/1320457783 >> 172.21.15.33:0/1320457783 conn(0x7f9e480d3db0 msgr2=0x7f9e480d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- 172.21.15.33:0/1320457783 shutdown_connections 2024-04-25T01:08:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- 172.21.15.33:0/1320457783 wait complete. 2024-04-25T01:08:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 Processor -- start 2024-04-25T01:08:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- start start 2024-04-25T01:08:47.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d03e0 0x7f9e48155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e480d48d0 0x7f9e480c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4815e470 0x7f9e4814cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e480d3310 con 0x7f9e480d03e0 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e480d3190 con 0x7f9e4815e470 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e480d3490 con 0x7f9e480d48d0 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d03e0 0x7f9e48155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d03e0 0x7f9e48155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50292/0 (socket says 172.21.15.33:50292) 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 -- 172.21.15.33:0/2174544192 learned_addr learned my addr 172.21.15.33:0/2174544192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 -- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e480d48d0 msgr2=0x7f9e480c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e480d48d0 0x7f9e480c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 -- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4815e470 msgr2=0x7f9e4814cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4815e470 0x7f9e4814cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 -- 172.21.15.33:0/2174544192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e480c9a50 con 0x7f9e480d03e0 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e46ffd640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d03e0 0x7f9e48155df0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800d5f0 tx=0x7f9e3800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e2f7fe640 1 -- 172.21.15.33:0/2174544192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380125f0 con 0x7f9e480d03e0 2024-04-25T01:08:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e2f7fe640 1 -- 172.21.15.33:0/2174544192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e38019070 con 0x7f9e480d03e0 2024-04-25T01:08:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e2f7fe640 1 -- 172.21.15.33:0/2174544192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380129b0 con 0x7f9e480d03e0 2024-04-25T01:08:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e480c9cf0 con 0x7f9e480d03e0 2024-04-25T01:08:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.421+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e480c8310 con 0x7f9e480d03e0 2024-04-25T01:08:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.425+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4814d690 con 0x7f9e480d03e0 2024-04-25T01:08:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.425+0000 7f9e2f7fe640 1 -- 172.21.15.33:0/2174544192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e38012b50 con 0x7f9e480d03e0 2024-04-25T01:08:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.425+0000 7f9e2f7fe640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e340678c0 0x7f9e34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.425+0000 7f9e2f7fe640 1 -- 172.21.15.33:0/2174544192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9e38086380 con 0x7f9e480d03e0 2024-04-25T01:08:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.425+0000 7f9e467fc640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e340678c0 0x7f9e34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.425+0000 7f9e467fc640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e340678c0 0x7f9e34069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007fd0 tx=0x7f9e3c007940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.429+0000 7f9e2f7fe640 1 -- 172.21.15.33:0/2174544192 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e38056c10 con 0x7f9e480d03e0 2024-04-25T01:08:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.585+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e48069040 con 0x7f9e480d03e0 2024-04-25T01:08:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e2f7fe640 1 -- 172.21.15.33:0/2174544192 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f9e38056510 con 0x7f9e480d03e0 2024-04-25T01:08:47.580 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:47.580 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e340678c0 msgr2=0x7f9e34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e340678c0 0x7f9e34069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007fd0 tx=0x7f9e3c007940 comp rx=0 tx=0).stop 2024-04-25T01:08:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d03e0 msgr2=0x7f9e48155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d03e0 0x7f9e48155df0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800d5f0 tx=0x7f9e3800dab0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 shutdown_connections 2024-04-25T01:08:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e340678c0 0x7f9e34069d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4815e470 0x7f9e4814cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e480d48d0 0x7f9e480c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 --2- 172.21.15.33:0/2174544192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e480d03e0 0x7f9e48155df0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 >> 172.21.15.33:0/2174544192 conn(0x7f9e480d3db0 msgr2=0x7f9e4815e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 shutdown_connections 2024-04-25T01:08:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.589+0000 7f9e4d41d640 1 -- 172.21.15.33:0/2174544192 wait complete. 2024-04-25T01:08:47.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:08:47.598 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:08:47.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5fc38640 1 Processor -- start 2024-04-25T01:08:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5fc38640 1 -- start start 2024-04-25T01:08:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5fc38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee58145660 0x7fee5806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5fc38640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee5815f960 0x7fee5815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5fc38640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee58160770 con 0x7fee581602e0 2024-04-25T01:08:47.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5fc38640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fee5806f0a0 con 0x7fee5815f960 2024-04-25T01:08:47.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5fc38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee58145a40 con 0x7fee58145660 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5d1d3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee5815f960 0x7fee5815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5d9d4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee58145660 0x7fee5806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5e1d5640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fee581602e0 0x7fee5806d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:48630/0 (socket says 172.21.15.33:48630) 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5e1d5640 1 -- 172.21.15.33:0/186476706 learned_addr learned my addr 172.21.15.33:0/186476706 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5c9d2640 1 -- 172.21.15.33:0/186476706 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2711899083 0 0) 0x7fee58160770 con 0x7fee581602e0 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5c9d2640 1 -- 172.21.15.33:0/186476706 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee40003530 con 0x7fee581602e0 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5d9d4640 1 -- 172.21.15.33:0/186476706 >> v2:172.21.15.138:3300/0 conn(0x7fee5815f960 msgr2=0x7fee5815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5d9d4640 1 --2- 172.21.15.33:0/186476706 >> v2:172.21.15.138:3300/0 conn(0x7fee5815f960 0x7fee5815fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5d9d4640 1 -- 172.21.15.33:0/186476706 >> v1:172.21.15.138:6789/0 conn(0x7fee581602e0 legacy=0x7fee5806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.721+0000 7fee5d9d4640 1 -- 172.21.15.33:0/186476706 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee580cee10 con 0x7fee58145660 2024-04-25T01:08:47.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5d9d4640 1 --2- 172.21.15.33:0/186476706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee58145660 0x7fee5806eb60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee48009d90 tx=0x7fee4801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ef96b8e1623641e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5c9d2640 1 -- 172.21.15.33:0/186476706 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee480b2070 con 0x7fee58145660 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5c9d2640 1 -- 172.21.15.33:0/186476706 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee480a5920 con 0x7fee58145660 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5c9d2640 1 -- 172.21.15.33:0/186476706 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee480a5cd0 con 0x7fee58145660 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 -- 172.21.15.33:0/186476706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee58145660 msgr2=0x7fee5806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 --2- 172.21.15.33:0/186476706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee58145660 0x7fee5806eb60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fee48009d90 tx=0x7fee4801c970 comp rx=0 tx=0).stop 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 -- 172.21.15.33:0/186476706 shutdown_connections 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 --2- 172.21.15.33:0/186476706 >> v2:172.21.15.138:3300/0 conn(0x7fee5815f960 0x7fee5815fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 --2- 172.21.15.33:0/186476706 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee58145660 0x7fee5806eb60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 -- 172.21.15.33:0/186476706 >> 172.21.15.33:0/186476706 conn(0x7fee580b1810 msgr2=0x7fee580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 -- 172.21.15.33:0/186476706 shutdown_connections 2024-04-25T01:08:47.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 -- 172.21.15.33:0/186476706 wait complete. 2024-04-25T01:08:47.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 Processor -- start 2024-04-25T01:08:47.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.725+0000 7fee5fc38640 1 -- start start 2024-04-25T01:08:47.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee58145660 0x7fee58134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee5815f960 0x7fee580b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee581602e0 0x7fee580b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee58168e20 con 0x7fee58145660 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee58168ca0 con 0x7fee581602e0 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee58168fa0 con 0x7fee5815f960 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee58145660 0x7fee58134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee58145660 0x7fee58134220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50304/0 (socket says 172.21.15.33:50304) 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 -- 172.21.15.33:0/1494926283 learned_addr learned my addr 172.21.15.33:0/1494926283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:47.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d1d3640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee5815f960 0x7fee580b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5e1d5640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee581602e0 0x7fee580b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 -- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee5815f960 msgr2=0x7fee580b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee5815f960 0x7fee580b3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 -- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee581602e0 msgr2=0x7fee580b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee581602e0 0x7fee580b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 -- 172.21.15.33:0/1494926283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5815a000 con 0x7fee58145660 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d9d4640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee58145660 0x7fee58134220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f90 tx=0x7fee48019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee567fc640 1 -- 172.21.15.33:0/1494926283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee480be020 con 0x7fee58145660 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee567fc640 1 -- 172.21.15.33:0/1494926283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee48003ba0 con 0x7fee58145660 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee58159d20 con 0x7fee58145660 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee567fc640 1 -- 172.21.15.33:0/1494926283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee480b2040 con 0x7fee58145660 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee5815c710 con 0x7fee58145660 2024-04-25T01:08:47.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee58069040 con 0x7fee58145660 2024-04-25T01:08:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee567fc640 1 -- 172.21.15.33:0/1494926283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fee480c1070 con 0x7fee58145660 2024-04-25T01:08:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee567fc640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee440678c0 0x7fee44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee567fc640 1 -- 172.21.15.33:0/1494926283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fee48120480 con 0x7fee58145660 2024-04-25T01:08:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.729+0000 7fee5d1d3640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee440678c0 0x7fee44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.733+0000 7fee5d1d3640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee440678c0 0x7fee44069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fee5815c2c0 tx=0x7fee4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.733+0000 7fee567fc640 1 -- 172.21.15.33:0/1494926283 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fee48016560 con 0x7fee58145660 2024-04-25T01:08:47.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.893+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee58072ac0 con 0x7fee58145660 2024-04-25T01:08:47.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.893+0000 7fee567fc640 1 -- 172.21.15.33:0/1494926283 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fee480f0d40 con 0x7fee58145660 2024-04-25T01:08:47.885 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:08:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.893+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee440678c0 msgr2=0x7fee44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.893+0000 7fee5fc38640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee440678c0 0x7fee44069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fee5815c2c0 tx=0x7fee4c008040 comp rx=0 tx=0).stop 2024-04-25T01:08:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.893+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee58145660 msgr2=0x7fee58134220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.893+0000 7fee5fc38640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee58145660 0x7fee58134220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fee48000f90 tx=0x7fee48019710 comp rx=0 tx=0).stop 2024-04-25T01:08:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 shutdown_connections 2024-04-25T01:08:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee440678c0 0x7fee44069d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee581602e0 0x7fee580b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee5815f960 0x7fee580b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 --2- 172.21.15.33:0/1494926283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee58145660 0x7fee58134220 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 >> 172.21.15.33:0/1494926283 conn(0x7fee580b1810 msgr2=0x7fee5815f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 shutdown_connections 2024-04-25T01:08:47.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:47.897+0000 7fee5fc38640 1 -- 172.21.15.33:0/1494926283 wait complete. 2024-04-25T01:08:47.903 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:08:48.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 Processor -- start 2024-04-25T01:08:48.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 -- start start 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8c3815e0d0 0x7f8c380cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c380d1f60 0x7f8c380d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c380b9a20 0x7f8c380d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8c380b7580 con 0x7f8c380b9a20 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8c380b6cc0 con 0x7f8c3815e0d0 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3e406640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c380b6f30 con 0x7f8c380d1f60 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3c9a3640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c380b9a20 0x7f8c380d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3c9a3640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8c380b9a20 0x7f8c380d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50318/0 (socket says 172.21.15.33:50318) 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c3c9a3640 1 -- 172.21.15.33:0/1578001187 learned_addr learned my addr 172.21.15.33:0/1578001187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.033+0000 7f8c2ffff640 1 --2- 172.21.15.33:0/1578001187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c380d1f60 0x7f8c380d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c37fff640 1 --2- 172.21.15.33:0/1578001187 >> v2:172.21.15.138:3300/0 conn(0x7f8c3815e0d0 0x7f8c380cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3c9a3640 1 -- 172.21.15.33:0/1578001187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c380d1f60 msgr2=0x7f8c380d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3c9a3640 1 --2- 172.21.15.33:0/1578001187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c380d1f60 0x7f8c380d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3c9a3640 1 -- 172.21.15.33:0/1578001187 >> v2:172.21.15.138:3300/0 conn(0x7f8c3815e0d0 msgr2=0x7f8c380cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3c9a3640 1 --2- 172.21.15.33:0/1578001187 >> v2:172.21.15.138:3300/0 conn(0x7f8c3815e0d0 0x7f8c380cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3c9a3640 1 -- 172.21.15.33:0/1578001187 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c380b7210 con 0x7f8c380b9a20 2024-04-25T01:08:48.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c37fff640 1 --2- 172.21.15.33:0/1578001187 >> v2:172.21.15.138:3300/0 conn(0x7f8c3815e0d0 0x7f8c380cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:48.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3c9a3640 1 --2- 172.21.15.33:0/1578001187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380b9a20 0x7f8c380d28f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c380d17b0 tx=0x7f8c3001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c789b6bf2655ed60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c377fe640 1 -- 172.21.15.33:0/1578001187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300a57a0 con 0x7f8c380b9a20 2024-04-25T01:08:48.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c377fe640 1 -- 172.21.15.33:0/1578001187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c300a5940 con 0x7f8c380b9a20 2024-04-25T01:08:48.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c377fe640 1 -- 172.21.15.33:0/1578001187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c300a5d10 con 0x7f8c380b9a20 2024-04-25T01:08:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 -- 172.21.15.33:0/1578001187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380b9a20 msgr2=0x7f8c380d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 --2- 172.21.15.33:0/1578001187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380b9a20 0x7f8c380d28f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8c380d17b0 tx=0x7f8c3001c9f0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 -- 172.21.15.33:0/1578001187 shutdown_connections 2024-04-25T01:08:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 --2- 172.21.15.33:0/1578001187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380b9a20 0x7f8c380d28f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 --2- 172.21.15.33:0/1578001187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c380d1f60 0x7f8c380d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 --2- 172.21.15.33:0/1578001187 >> v2:172.21.15.138:3300/0 conn(0x7f8c3815e0d0 0x7f8c380cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 -- 172.21.15.33:0/1578001187 >> 172.21.15.33:0/1578001187 conn(0x7f8c380d3dd0 msgr2=0x7f8c380d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 -- 172.21.15.33:0/1578001187 shutdown_connections 2024-04-25T01:08:48.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.037+0000 7f8c3e406640 1 -- 172.21.15.33:0/1578001187 wait complete. 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 Processor -- start 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 -- start start 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c380b9a20 0x7f8c38156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380d1f60 0x7f8c380cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c3815e0d0 0x7f8c3815ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c380d10c0 con 0x7f8c380d1f60 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8c380d0f40 con 0x7f8c380b9a20 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8c380d1240 con 0x7f8c3815e0d0 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380d1f60 0x7f8c380cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380d1f60 0x7f8c380cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50322/0 (socket says 172.21.15.33:50322) 2024-04-25T01:08:48.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 -- 172.21.15.33:0/3939581129 learned_addr learned my addr 172.21.15.33:0/3939581129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c37fff640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c380b9a20 0x7f8c38156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3c9a3640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c3815e0d0 0x7f8c3815ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 -- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c3815e0d0 msgr2=0x7f8c3815ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c3815e0d0 0x7f8c3815ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 -- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c380b9a20 msgr2=0x7f8c38156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c380b9a20 0x7f8c38156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 -- 172.21.15.33:0/3939581129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c380cb410 con 0x7f8c380d1f60 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c2ffff640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380d1f60 0x7f8c380cc4b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000d800 tx=0x7f8c2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c357fa640 1 -- 172.21.15.33:0/3939581129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000b6c0 con 0x7f8c380d1f60 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c357fa640 1 -- 172.21.15.33:0/3939581129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c20019070 con 0x7f8c380d1f60 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c357fa640 1 -- 172.21.15.33:0/3939581129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000baa0 con 0x7f8c380d1f60 2024-04-25T01:08:48.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c380cb6f0 con 0x7f8c380d1f60 2024-04-25T01:08:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c380cbcb0 con 0x7f8c380d1f60 2024-04-25T01:08:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c38069040 con 0x7f8c380d1f60 2024-04-25T01:08:48.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.041+0000 7f8c357fa640 1 -- 172.21.15.33:0/3939581129 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c2000bc40 con 0x7f8c380d1f60 2024-04-25T01:08:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.045+0000 7f8c357fa640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8c180678c0 0x7f8c18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.045+0000 7f8c357fa640 1 -- 172.21.15.33:0/3939581129 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8c20086510 con 0x7f8c380d1f60 2024-04-25T01:08:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.045+0000 7f8c37fff640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8c180678c0 0x7f8c18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.045+0000 7f8c37fff640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8c180678c0 0x7f8c18069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8c28006fd0 tx=0x7f8c28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.049+0000 7f8c357fa640 1 -- 172.21.15.33:0/3939581129 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8c20056c70 con 0x7f8c380d1f60 2024-04-25T01:08:48.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.201+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8c380c7510 con 0x7f8c380d1f60 2024-04-25T01:08:48.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.201+0000 7f8c357fa640 1 -- 172.21.15.33:0/3939581129 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f8c20056570 con 0x7f8c380d1f60 2024-04-25T01:08:48.195 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8c180678c0 msgr2=0x7f8c18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8c180678c0 0x7f8c18069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8c28006fd0 tx=0x7f8c28008040 comp rx=0 tx=0).stop 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380d1f60 msgr2=0x7f8c380cc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380d1f60 0x7f8c380cc4b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000d800 tx=0x7f8c2000dcc0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 shutdown_connections 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8c180678c0 0x7f8c18069d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8c3815e0d0 0x7f8c3815ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c380d1f60 0x7f8c380cc4b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 --2- 172.21.15.33:0/3939581129 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8c380b9a20 0x7f8c38156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 >> 172.21.15.33:0/3939581129 conn(0x7f8c380d3dd0 msgr2=0x7f8c380d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 shutdown_connections 2024-04-25T01:08:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.205+0000 7f8c3e406640 1 -- 172.21.15.33:0/3939581129 wait complete. 2024-04-25T01:08:48.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:08:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f99a4cb2640 1 Processor -- start 2024-04-25T01:08:48.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f99a4cb2640 1 -- start start 2024-04-25T01:08:48.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f99a4cb2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 0x7f99a006c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f99a4cb2640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a00b4000 con 0x7f99a01583d0 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f99a4cb2640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99a00b42e0 con 0x7f99a006c760 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f99a4cb2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f99a006cbd0 con 0x7f99a01587b0 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999e59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f99a01583d0 0x7f99a00720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:34016/0 (socket says 172.21.15.33:34016) 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999e59c640 1 -- 172.21.15.33:0/3871719412 learned_addr learned my addr 172.21.15.33:0/3871719412 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999dd9b640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 0x7f99a006c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1187165957 0 0) 0x7f99a00b4000 con 0x7f99a01583d0 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f998c003530 con 0x7f99a01583d0 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3827990858 0 0) 0x7f998c003530 con 0x7f99a01583d0 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 msgr2=0x7f99a006c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:48.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 0x7f99a006c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 >> v1:172.21.15.138:6789/0 conn(0x7f99a006c760 legacy=0x7f99a00ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a00b45c0 con 0x7f99a01583d0 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f9990001680 con 0x7f99a01583d0 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 >> v1:172.21.15.33:6789/0 conn(0x7f99a01583d0 legacy=0x7f99a00720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 0x7f998c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c003f70 0x7f998c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c004d50 0x7f998c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a006cbd0 con 0x7f998c003f70 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f998c00bab0 con 0x7f998c004d50 2024-04-25T01:08:48.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f998c00bcb0 con 0x7f99a01587b0 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999dd9b640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c003f70 0x7f998c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999e59c640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 0x7f998c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999ed9d640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c004d50 0x7f998c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999dd9b640 1 -- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 msgr2=0x7f998c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999dd9b640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a01587b0 0x7f998c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999dd9b640 1 -- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c004d50 msgr2=0x7f998c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999dd9b640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c004d50 0x7f998c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999dd9b640 1 -- 172.21.15.33:0/3871719412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998c003530 con 0x7f998c003f70 2024-04-25T01:08:48.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.345+0000 7f999ed9d640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c004d50 0x7f998c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f999dd9b640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c003f70 0x7f998c004810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fa0 tx=0x7f998801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76133b5778eeaf27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99880b2070 con 0x7f998c003f70 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99880a5c00 con 0x7f998c003f70 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f999d59a640 1 -- 172.21.15.33:0/3871719412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9988009bf0 con 0x7f998c003f70 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- 172.21.15.33:0/3871719412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c003f70 msgr2=0x7f998c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 --2- 172.21.15.33:0/3871719412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f998c003f70 0x7f998c004810 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9988009fa0 tx=0x7f998801cc40 comp rx=0 tx=0).stop 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f999ed9d640 1 -- 172.21.15.33:0/3871719412 reap_dead start 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- 172.21.15.33:0/3871719412 shutdown_connections 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- 172.21.15.33:0/3871719412 >> 172.21.15.33:0/3871719412 conn(0x7f99a00715d0 msgr2=0x7f99a00719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- 172.21.15.33:0/3871719412 shutdown_connections 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- 172.21.15.33:0/3871719412 wait complete. 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 Processor -- start 2024-04-25T01:08:48.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- start start 2024-04-25T01:08:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c003f70 0x7f99a0155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a0155a50 0x7f99a0157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a0157880 0x7f99a015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0072800 con 0x7f99a0157880 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f99a00b4c10 con 0x7f998c003f70 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f99a4cb2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f99a00afcc0 con 0x7f99a0155a50 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.349+0000 7f999ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a0157880 0x7f99a015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a0157880 0x7f99a015bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50348/0 (socket says 172.21.15.33:50348) 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 -- 172.21.15.33:0/2813687012 learned_addr learned my addr 172.21.15.33:0/2813687012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999dd9b640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a0155a50 0x7f99a0157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999e59c640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c003f70 0x7f99a0155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 -- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a0155a50 msgr2=0x7f99a0157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a0155a50 0x7f99a0157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 -- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c003f70 msgr2=0x7f99a0155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c003f70 0x7f99a0155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 -- 172.21.15.33:0/2813687012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a01541d0 con 0x7f99a0157880 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999e59c640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c003f70 0x7f99a0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999ed9d640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a0157880 0x7f99a015bf50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99a00739e0 tx=0x7f999800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f9996ffd640 1 -- 172.21.15.33:0/2813687012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998005480 con 0x7f99a0157880 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f9996ffd640 1 -- 172.21.15.33:0/2813687012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9998007030 con 0x7f99a0157880 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a0154470 con 0x7f99a0157880 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f9996ffd640 1 -- 172.21.15.33:0/2813687012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998007400 con 0x7f99a0157880 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a0154a30 con 0x7f99a0157880 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a0069040 con 0x7f99a0157880 2024-04-25T01:08:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f9996ffd640 1 -- 172.21.15.33:0/2813687012 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99980067e0 con 0x7f99a0157880 2024-04-25T01:08:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f9996ffd640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f99800678f0 0x7f9980069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f9996ffd640 1 -- 172.21.15.33:0/2813687012 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9998085920 con 0x7f99a0157880 2024-04-25T01:08:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999e59c640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f99800678f0 0x7f9980069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.353+0000 7f999e59c640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f99800678f0 0x7f9980069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9990005380 tx=0x7f9990001530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.357+0000 7f9996ffd640 1 -- 172.21.15.33:0/2813687012 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f99980561b0 con 0x7f99a0157880 2024-04-25T01:08:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.521+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f99a0141530 con 0x7f99a0157880 2024-04-25T01:08:48.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.521+0000 7f9996ffd640 1 -- 172.21.15.33:0/2813687012 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f9998055ab0 con 0x7f99a0157880 2024-04-25T01:08:48.514 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:08:48.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f99800678f0 msgr2=0x7f9980069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f99800678f0 0x7f9980069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9990005380 tx=0x7f9990001530 comp rx=0 tx=0).stop 2024-04-25T01:08:48.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a0157880 msgr2=0x7f99a015bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a0157880 0x7f99a015bf50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f99a00739e0 tx=0x7f999800d890 comp rx=0 tx=0).stop 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 shutdown_connections 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f99800678f0 0x7f9980069db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99a0157880 0x7f99a015bf50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f99a0155a50 0x7f99a0157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 --2- 172.21.15.33:0/2813687012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f998c003f70 0x7f99a0155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 >> 172.21.15.33:0/2813687012 conn(0x7f99a00715d0 msgr2=0x7f99a015f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 shutdown_connections 2024-04-25T01:08:48.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.525+0000 7f99a4cb2640 1 -- 172.21.15.33:0/2813687012 wait complete. 2024-04-25T01:08:48.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:08:48.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.653+0000 7f46488d2640 1 Processor -- start 2024-04-25T01:08:48.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.653+0000 7f46488d2640 1 -- start start 2024-04-25T01:08:48.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f46488d2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f464415e0d0 0x7f46440cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f46488d2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440d1f60 0x7f46440d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f46488d2640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46440b6cc0 con 0x7f46440b9a20 2024-04-25T01:08:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f46488d2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f46440b7580 con 0x7f464415e0d0 2024-04-25T01:08:48.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f46488d2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f46440b6f30 con 0x7f46440d1f60 2024-04-25T01:08:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464259c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f464415e0d0 0x7f46440cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464259c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f464415e0d0 0x7f46440cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:46370/0 (socket says 172.21.15.33:46370) 2024-04-25T01:08:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f4641d9b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440d1f60 0x7f46440d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464259c640 1 -- 172.21.15.33:0/1009893183 learned_addr learned my addr 172.21.15.33:0/1009893183 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f4642d9d640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f46440b9a20 0x7f46440d0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:58438/0 (socket says 172.21.15.33:58438) 2024-04-25T01:08:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464259c640 1 -- 172.21.15.33:0/1009893183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440d1f60 msgr2=0x7f46440d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:48.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464259c640 1 --2- 172.21.15.33:0/1009893183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440d1f60 0x7f46440d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464259c640 1 -- 172.21.15.33:0/1009893183 >> v1:172.21.15.138:6789/0 conn(0x7f46440b9a20 legacy=0x7f46440d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464159a640 1 -- 172.21.15.33:0/1009893183 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075954327 0 0) 0x7f46440b6cc0 con 0x7f46440b9a20 2024-04-25T01:08:48.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.657+0000 7f464259c640 1 -- 172.21.15.33:0/1009893183 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46440b7210 con 0x7f464415e0d0 2024-04-25T01:08:48.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f464259c640 1 --2- 172.21.15.33:0/1009893183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 0x7f46440cf3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4634009870 tx=0x7f463401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a306a3ff85d9dac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f464159a640 1 -- 172.21.15.33:0/1009893183 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46340a7070 con 0x7f464415e0d0 2024-04-25T01:08:48.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f464159a640 1 -- 172.21.15.33:0/1009893183 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46340b0070 con 0x7f464415e0d0 2024-04-25T01:08:48.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f464159a640 1 -- 172.21.15.33:0/1009893183 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46340ab890 con 0x7f464415e0d0 2024-04-25T01:08:48.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 -- 172.21.15.33:0/1009893183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 msgr2=0x7f46440cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 --2- 172.21.15.33:0/1009893183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 0x7f46440cf3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4634009870 tx=0x7f463401c560 comp rx=0 tx=0).stop 2024-04-25T01:08:48.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 -- 172.21.15.33:0/1009893183 shutdown_connections 2024-04-25T01:08:48.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 --2- 172.21.15.33:0/1009893183 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440d1f60 0x7f46440d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 --2- 172.21.15.33:0/1009893183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 0x7f46440cf3f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 -- 172.21.15.33:0/1009893183 >> 172.21.15.33:0/1009893183 conn(0x7f46440d3dd0 msgr2=0x7f46440d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 -- 172.21.15.33:0/1009893183 shutdown_connections 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.661+0000 7f46488d2640 1 -- 172.21.15.33:0/1009893183 wait complete. 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 Processor -- start 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 -- start start 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440b9a20 0x7f4644154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46440d1f60 0x7f464406c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 0x7f464406ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f464416ae90 con 0x7f46440d1f60 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f464416ad10 con 0x7f464415e0d0 2024-04-25T01:08:48.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f464416b010 con 0x7f46440b9a20 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46440d1f60 0x7f464406c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46440d1f60 0x7f464406c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59334/0 (socket says 172.21.15.33:59334) 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 -- 172.21.15.33:0/3412787073 learned_addr learned my addr 172.21.15.33:0/3412787073 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4642d9d640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 0x7f464406ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f464259c640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440b9a20 0x7f4644154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 -- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440b9a20 msgr2=0x7f4644154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440b9a20 0x7f4644154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 -- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 msgr2=0x7f464406ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 0x7f464406ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 -- 172.21.15.33:0/3412787073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46440c89d0 con 0x7f46440d1f60 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4641d9b640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46440d1f60 0x7f464406c530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f463800e900 tx=0x7f463800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4632ffd640 1 -- 172.21.15.33:0/3412787073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638017070 con 0x7f46440d1f60 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46440c8cb0 con 0x7f46440d1f60 2024-04-25T01:08:48.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4632ffd640 1 -- 172.21.15.33:0/3412787073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f463800ca00 con 0x7f46440d1f60 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4632ffd640 1 -- 172.21.15.33:0/3412787073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463800cdd0 con 0x7f46440d1f60 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46440c9270 con 0x7f46440d1f60 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4644069040 con 0x7f46440d1f60 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.665+0000 7f4632ffd640 1 -- 172.21.15.33:0/3412787073 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f463801a070 con 0x7f46440d1f60 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.669+0000 7f4632ffd640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46240678c0 0x7f4624069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.669+0000 7f4632ffd640 1 -- 172.21.15.33:0/3412787073 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4638085fe0 con 0x7f46440d1f60 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.669+0000 7f464259c640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46240678c0 0x7f4624069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.669+0000 7f464259c640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46240678c0 0x7f4624069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f46340af000 tx=0x7f46340a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.673+0000 7f4632ffd640 1 -- 172.21.15.33:0/3412787073 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4638056160 con 0x7f46440d1f60 2024-04-25T01:08:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.825+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f46440ca640 con 0x7f46440d1f60 2024-04-25T01:08:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.825+0000 7f4632ffd640 1 -- 172.21.15.33:0/3412787073 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f463805a460 con 0x7f46440d1f60 2024-04-25T01:08:48.819 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46240678c0 msgr2=0x7f4624069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46240678c0 0x7f4624069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f46340af000 tx=0x7f46340a7000 comp rx=0 tx=0).stop 2024-04-25T01:08:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46440d1f60 msgr2=0x7f464406c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46440d1f60 0x7f464406c530 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f463800e900 tx=0x7f463800edc0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 shutdown_connections 2024-04-25T01:08:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46240678c0 0x7f4624069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f464415e0d0 0x7f464406ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46440d1f60 0x7f464406c530 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 --2- 172.21.15.33:0/3412787073 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46440b9a20 0x7f4644154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 >> 172.21.15.33:0/3412787073 conn(0x7f46440d3dd0 msgr2=0x7f46440b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 shutdown_connections 2024-04-25T01:08:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.829+0000 7f46488d2640 1 -- 172.21.15.33:0/3412787073 wait complete. 2024-04-25T01:08:48.837 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:48.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.965+0000 7f6a59c34640 1 Processor -- start 2024-04-25T01:08:48.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.965+0000 7f6a59c34640 1 -- start start 2024-04-25T01:08:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a540b5530 0x7f6a54166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a5415f9a0 con 0x7f6a5415c160 2024-04-25T01:08:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a5415fc80 con 0x7f6a54145470 2024-04-25T01:08:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6a5415f750 con 0x7f6a540b5530 2024-04-25T01:08:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a53fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a540b5530 0x7f6a54166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a4affd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6a5415c160 0x7f6a540b4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51258/0 (socket says 172.21.15.33:51258) 2024-04-25T01:08:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a4affd640 1 -- 172.21.15.33:0/3682515463 learned_addr learned my addr 172.21.15.33:0/3682515463 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a53fff640 1 -- 172.21.15.33:0/3682515463 >> v1:172.21.15.138:6789/0 conn(0x7f6a54145470 legacy=0x7f6a540cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a53fff640 1 -- 172.21.15.33:0/3682515463 >> v1:172.21.15.33:6789/0 conn(0x7f6a5415c160 legacy=0x7f6a540b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a52ffd640 1 -- 172.21.15.33:0/3682515463 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239529848 0 0) 0x7f6a5415f9a0 con 0x7f6a5415c160 2024-04-25T01:08:48.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a53fff640 1 -- 172.21.15.33:0/3682515463 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5415ff60 con 0x7f6a540b5530 2024-04-25T01:08:48.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a53fff640 1 --2- 172.21.15.33:0/3682515463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a540b5530 0x7f6a54166ae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009d90 tx=0x7f6a4c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2305fcbc304df88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a52ffd640 1 -- 172.21.15.33:0/3682515463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0b2070 con 0x7f6a540b5530 2024-04-25T01:08:48.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a52ffd640 1 -- 172.21.15.33:0/3682515463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0a5900 con 0x7f6a540b5530 2024-04-25T01:08:48.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a52ffd640 1 -- 172.21.15.33:0/3682515463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0a5cd0 con 0x7f6a540b5530 2024-04-25T01:08:48.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- 172.21.15.33:0/3682515463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a540b5530 msgr2=0x7f6a54166ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 --2- 172.21.15.33:0/3682515463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a540b5530 0x7f6a54166ae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009d90 tx=0x7f6a4c01c930 comp rx=0 tx=0).stop 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- 172.21.15.33:0/3682515463 shutdown_connections 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 --2- 172.21.15.33:0/3682515463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a540b5530 0x7f6a54166ae0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- 172.21.15.33:0/3682515463 >> 172.21.15.33:0/3682515463 conn(0x7f6a540b1810 msgr2=0x7f6a540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- 172.21.15.33:0/3682515463 shutdown_connections 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.969+0000 7f6a59c34640 1 -- 172.21.15.33:0/3682515463 wait complete. 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 Processor -- start 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 -- start start 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a540b5530 0x7f6a54142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a54145470 0x7f6a54140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a5415c160 0x7f6a54139ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a54169840 con 0x7f6a5415c160 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5406e850 con 0x7f6a54145470 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6a541699c0 con 0x7f6a540b5530 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a5415c160 0x7f6a54139ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a5415c160 0x7f6a54139ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59356/0 (socket says 172.21.15.33:59356) 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 -- 172.21.15.33:0/1233143381 learned_addr learned my addr 172.21.15.33:0/1233143381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a4affd640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a54145470 0x7f6a54140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 -- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a540b5530 msgr2=0x7f6a54142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a540b5530 0x7f6a54142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 -- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a54145470 msgr2=0x7f6a54140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a54145470 0x7f6a54140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 -- 172.21.15.33:0/1233143381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a54142eb0 con 0x7f6a5415c160 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a4affd640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a54145470 0x7f6a54140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a53fff640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a5415c160 0x7f6a54139ec0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009d90 tx=0x7f6a4c0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a50ff9640 1 -- 172.21.15.33:0/1233143381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0b2050 con 0x7f6a5415c160 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a50ff9640 1 -- 172.21.15.33:0/1233143381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a4c0b3040 con 0x7f6a5415c160 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a54143190 con 0x7f6a5415c160 2024-04-25T01:08:48.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a50ff9640 1 -- 172.21.15.33:0/1233143381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c003dc0 con 0x7f6a5415c160 2024-04-25T01:08:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5413e270 con 0x7f6a5415c160 2024-04-25T01:08:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.973+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5413e470 con 0x7f6a5415c160 2024-04-25T01:08:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.977+0000 7f6a50ff9640 1 -- 172.21.15.33:0/1233143381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a4c0c1060 con 0x7f6a5415c160 2024-04-25T01:08:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.977+0000 7f6a50ff9640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a340678f0 0x7f6a34069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.977+0000 7f6a50ff9640 1 -- 172.21.15.33:0/1233143381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6a4c11ff70 con 0x7f6a5415c160 2024-04-25T01:08:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.977+0000 7f6a537fe640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a340678f0 0x7f6a34069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:48.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.977+0000 7f6a537fe640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a340678f0 0x7f6a34069db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a44003cf0 tx=0x7f6a440079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:48.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:48.981+0000 7f6a50ff9640 1 -- 172.21.15.33:0/1233143381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6a5413e470 con 0x7f6a5415c160 2024-04-25T01:08:49.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.177+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a54069040 con 0x7f6a5415c160 2024-04-25T01:08:49.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.177+0000 7f6a50ff9640 1 -- 172.21.15.33:0/1233143381 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f6a4c016520 con 0x7f6a5415c160 2024-04-25T01:08:49.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:49.171 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:08:49.171 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:08:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a340678f0 msgr2=0x7f6a34069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a340678f0 0x7f6a34069db0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6a44003cf0 tx=0x7f6a440079c0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a5415c160 msgr2=0x7f6a54139ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a5415c160 0x7f6a54139ec0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009d90 tx=0x7f6a4c0196d0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 shutdown_connections 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a340678f0 0x7f6a34069db0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a5415c160 0x7f6a54139ec0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a54145470 0x7f6a54140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 --2- 172.21.15.33:0/1233143381 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a540b5530 0x7f6a54142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 >> 172.21.15.33:0/1233143381 conn(0x7f6a540b1810 msgr2=0x7f6a540cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 shutdown_connections 2024-04-25T01:08:49.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.181+0000 7f6a59c34640 1 -- 172.21.15.33:0/1233143381 wait complete. 2024-04-25T01:08:49.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:49.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8da0c3640 1 Processor -- start 2024-04-25T01:08:49.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8da0c3640 1 -- start start 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8da0c3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb8d41456f0 0x7fb8d40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8da0c3640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d40725e0 con 0x7fb8d415ce40 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8da0c3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d40728c0 con 0x7fb8d406b920 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8da0c3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d4072ba0 con 0x7fb8d41456f0 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d2ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb8d415ce40 0x7fb8d406b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51264/0 (socket says 172.21.15.33:51264) 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d2ffd640 1 -- 172.21.15.33:0/3267881190 learned_addr learned my addr 172.21.15.33:0/3267881190 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/3267881190 >> v2:172.21.15.138:3300/0 conn(0x7fb8d41456f0 0x7fb8d40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353819895 0 0) 0x7fb8d40725e0 con 0x7fb8d415ce40 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8bc003530 con 0x7fb8d415ce40 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1180649886 0 0) 0x7fb8bc003530 con 0x7fb8d415ce40 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 >> v2:172.21.15.138:3300/0 conn(0x7fb8d41456f0 msgr2=0x7fb8d40cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 --2- 172.21.15.33:0/3267881190 >> v2:172.21.15.138:3300/0 conn(0x7fb8d41456f0 0x7fb8d40cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 >> v1:172.21.15.138:6789/0 conn(0x7fb8d406b920 legacy=0x7fb8d406ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d406e540 con 0x7fb8d415ce40 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fb8c8001680 con 0x7fb8d415ce40 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 >> v1:172.21.15.33:6789/0 conn(0x7fb8d415ce40 legacy=0x7fb8d406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc004810 0x7fb8bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8bc0051d0 0x7fb8bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d4072ba0 con 0x7fb8bc0039c0 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8bc00c070 con 0x7fb8bc004810 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb8bc00c270 con 0x7fb8bc0051d0 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d2ffd640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc004810 0x7fb8bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d3fff640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8bc0051d0 0x7fb8bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d37fe640 1 -- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8bc0051d0 msgr2=0x7fb8bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8bc0051d0 0x7fb8bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d37fe640 1 -- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc004810 msgr2=0x7fb8bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8bc004810 0x7fb8bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d37fe640 1 -- 172.21.15.33:0/3267881190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8bc003530 con 0x7fb8bc0039c0 2024-04-25T01:08:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.321+0000 7fb8d3fff640 1 -- 172.21.15.33:0/3267881190 reap_dead start 2024-04-25T01:08:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8bc003de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ae0 tx=0x7fb8c401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb35d06aa748ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40b2070 con 0x7fb8bc0039c0 2024-04-25T01:08:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c40a5550 con 0x7fb8bc0039c0 2024-04-25T01:08:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8d27fc640 1 -- 172.21.15.33:0/3267881190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40096b0 con 0x7fb8bc0039c0 2024-04-25T01:08:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 -- 172.21.15.33:0/3267881190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 msgr2=0x7fb8bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8bc003de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ae0 tx=0x7fb8c401ca70 comp rx=0 tx=0).stop 2024-04-25T01:08:49.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 -- 172.21.15.33:0/3267881190 shutdown_connections 2024-04-25T01:08:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/3267881190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8bc003de0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 -- 172.21.15.33:0/3267881190 >> 172.21.15.33:0/3267881190 conn(0x7fb8d40b2820 msgr2=0x7fb8d40b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 -- 172.21.15.33:0/3267881190 shutdown_connections 2024-04-25T01:08:49.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 -- 172.21.15.33:0/3267881190 wait complete. 2024-04-25T01:08:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 Processor -- start 2024-04-25T01:08:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.325+0000 7fb8da0c3640 1 -- start start 2024-04-25T01:08:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8d413f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8d41e7910 0x7fb8d4157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d41fae10 0x7fb8d41fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d406b5a0 con 0x7fb8bc0039c0 2024-04-25T01:08:49.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d406b420 con 0x7fb8d41fae10 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d406b720 con 0x7fb8d41e7910 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8d413f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8d413f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59372/0 (socket says 172.21.15.33:59372) 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 -- 172.21.15.33:0/1709361581 learned_addr learned my addr 172.21.15.33:0/1709361581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d3fff640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d41fae10 0x7fb8d41fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d2ffd640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8d41e7910 0x7fb8d4157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 -- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8d41e7910 msgr2=0x7fb8d4157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8d41e7910 0x7fb8d4157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 -- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d41fae10 msgr2=0x7fb8d41fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d41fae10 0x7fb8d41fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 -- 172.21.15.33:0/1709361581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d4141d10 con 0x7fb8bc0039c0 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d3fff640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d41fae10 0x7fb8d41fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8d37fe640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8d413f770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ae0 tx=0x7fb8bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8bbfff640 1 -- 172.21.15.33:0/1709361581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40bc020 con 0x7fb8bc0039c0 2024-04-25T01:08:49.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8bbfff640 1 -- 172.21.15.33:0/1709361581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4004020 con 0x7fb8bc0039c0 2024-04-25T01:08:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d4141ff0 con 0x7fb8bc0039c0 2024-04-25T01:08:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8bbfff640 1 -- 172.21.15.33:0/1709361581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40b2040 con 0x7fb8bc0039c0 2024-04-25T01:08:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d41425b0 con 0x7fb8bc0039c0 2024-04-25T01:08:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d4069040 con 0x7fb8bc0039c0 2024-04-25T01:08:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8bbfff640 1 -- 172.21.15.33:0/1709361581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb8c4003620 con 0x7fb8bc0039c0 2024-04-25T01:08:49.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8bbfff640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8c00678c0 0x7fb8c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.329+0000 7fb8bbfff640 1 -- 172.21.15.33:0/1709361581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb8c41201d0 con 0x7fb8bc0039c0 2024-04-25T01:08:49.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.333+0000 7fb8d2ffd640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8c00678c0 0x7fb8c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.333+0000 7fb8d2ffd640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8c00678c0 0x7fb8c0069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8001ec0 tx=0x7fb8c8000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.337+0000 7fb8bbfff640 1 -- 172.21.15.33:0/1709361581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb8c40c6020 con 0x7fb8bc0039c0 2024-04-25T01:08:49.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.497+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8d406c010 con 0x7fb8bc0039c0 2024-04-25T01:08:49.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.497+0000 7fb8bbfff640 1 -- 172.21.15.33:0/1709361581 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fb8c4016780 con 0x7fb8bc0039c0 2024-04-25T01:08:49.491 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:49.491 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8c00678c0 msgr2=0x7fb8c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8c00678c0 0x7fb8c0069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8001ec0 tx=0x7fb8c8000ee0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 msgr2=0x7fb8d413f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8d413f770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ae0 tx=0x7fb8bc005840 comp rx=0 tx=0).stop 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 shutdown_connections 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8c00678c0 0x7fb8c0069d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8d41fae10 0x7fb8d41fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8d41e7910 0x7fb8d4157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 --2- 172.21.15.33:0/1709361581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8bc0039c0 0x7fb8d413f770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 >> 172.21.15.33:0/1709361581 conn(0x7fb8d40b2820 msgr2=0x7fb8d40cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:49.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 shutdown_connections 2024-04-25T01:08:49.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.501+0000 7fb8da0c3640 1 -- 172.21.15.33:0/1709361581 wait complete. 2024-04-25T01:08:49.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:08:49.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:08:49.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6e425640 1 Processor -- start 2024-04-25T01:08:49.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6e425640 1 -- start start 2024-04-25T01:08:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6e425640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6815a3a0 0x7f9f680cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6e425640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f6815df40 0x7f9f6815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6e425640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f680b3560 con 0x7f9f6815eb90 2024-04-25T01:08:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6e425640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9f680b4450 con 0x7f9f6815df40 2024-04-25T01:08:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6e425640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f680cf8c0 con 0x7f9f6815a3a0 2024-04-25T01:08:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f67fff640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6815a3a0 0x7f9f680cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f6c9c2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9f6815df40 0x7f9f6815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f677fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f9f6815eb90 0x7f9f680b48e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:58472/0 (socket says 172.21.15.33:58472) 2024-04-25T01:08:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f677fe640 1 -- 172.21.15.33:0/994233546 learned_addr learned my addr 172.21.15.33:0/994233546 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f67fff640 1 -- 172.21.15.33:0/994233546 >> v2:172.21.15.138:3300/0 conn(0x7f9f6815df40 msgr2=0x7f9f6815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f67fff640 1 --2- 172.21.15.33:0/994233546 >> v2:172.21.15.138:3300/0 conn(0x7f9f6815df40 0x7f9f6815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f67fff640 1 -- 172.21.15.33:0/994233546 >> v1:172.21.15.138:6789/0 conn(0x7f9f6815eb90 legacy=0x7f9f680b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.633+0000 7f9f67fff640 1 -- 172.21.15.33:0/994233546 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f680b3840 con 0x7f9f6815a3a0 2024-04-25T01:08:49.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f67fff640 1 --2- 172.21.15.33:0/994233546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6815a3a0 0x7f9f680cf380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f580097c0 tx=0x7f9f5801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c65a15f36f6f013 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f66ffd640 1 -- 172.21.15.33:0/994233546 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580b2070 con 0x7f9f6815a3a0 2024-04-25T01:08:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f66ffd640 1 -- 172.21.15.33:0/994233546 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f580ad640 con 0x7f9f6815a3a0 2024-04-25T01:08:49.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f66ffd640 1 -- 172.21.15.33:0/994233546 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f580ada10 con 0x7f9f6815a3a0 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 -- 172.21.15.33:0/994233546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6815a3a0 msgr2=0x7f9f680cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 --2- 172.21.15.33:0/994233546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6815a3a0 0x7f9f680cf380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f580097c0 tx=0x7f9f5801c4a0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 -- 172.21.15.33:0/994233546 shutdown_connections 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 --2- 172.21.15.33:0/994233546 >> v2:172.21.15.138:3300/0 conn(0x7f9f6815df40 0x7f9f6815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 --2- 172.21.15.33:0/994233546 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f6815a3a0 0x7f9f680cf380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 -- 172.21.15.33:0/994233546 >> 172.21.15.33:0/994233546 conn(0x7f9f680b1810 msgr2=0x7f9f680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 -- 172.21.15.33:0/994233546 shutdown_connections 2024-04-25T01:08:49.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 -- 172.21.15.33:0/994233546 wait complete. 2024-04-25T01:08:49.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 Processor -- start 2024-04-25T01:08:49.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.637+0000 7f9f6e425640 1 -- start start 2024-04-25T01:08:49.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f6815a3a0 0x7f9f681f7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f6815df40 0x7f9f68158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f680c4af0 0x7f9f68144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6806d420 con 0x7f9f6815df40 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f6815b790 con 0x7f9f6815a3a0 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6806d5a0 con 0x7f9f680c4af0 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f6815df40 0x7f9f68158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f6815df40 0x7f9f68158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59386/0 (socket says 172.21.15.33:59386) 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 -- 172.21.15.33:0/229426574 learned_addr learned my addr 172.21.15.33:0/229426574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f67fff640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f6815a3a0 0x7f9f681f7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6c9c2640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f680c4af0 0x7f9f68144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 -- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f680c4af0 msgr2=0x7f9f68144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f680c4af0 0x7f9f68144050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 -- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f6815a3a0 msgr2=0x7f9f681f7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f6815a3a0 0x7f9f681f7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 -- 172.21.15.33:0/229426574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f681f8e40 con 0x7f9f6815df40 2024-04-25T01:08:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6c9c2640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f680c4af0 0x7f9f68144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f677fe640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f6815df40 0x7f9f68158430 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00c370 tx=0x7f9f5c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f64ff9640 1 -- 172.21.15.33:0/229426574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c017070 con 0x7f9f6815df40 2024-04-25T01:08:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f64ff9640 1 -- 172.21.15.33:0/229426574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c005cf0 con 0x7f9f6815df40 2024-04-25T01:08:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f681f8880 con 0x7f9f6815df40 2024-04-25T01:08:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f64ff9640 1 -- 172.21.15.33:0/229426574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c0072f0 con 0x7f9f6815df40 2024-04-25T01:08:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f68162580 con 0x7f9f6815df40 2024-04-25T01:08:49.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f68069040 con 0x7f9f6815df40 2024-04-25T01:08:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f64ff9640 1 -- 172.21.15.33:0/229426574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f5c01a070 con 0x7f9f6815df40 2024-04-25T01:08:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f64ff9640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9f540678c0 0x7f9f54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f64ff9640 1 -- 172.21.15.33:0/229426574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9f5c085350 con 0x7f9f6815df40 2024-04-25T01:08:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.641+0000 7f9f67fff640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9f540678c0 0x7f9f54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.645+0000 7f9f67fff640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9f540678c0 0x7f9f54069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f580af000 tx=0x7f9f58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.645+0000 7f9f64ff9640 1 -- 172.21.15.33:0/229426574 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9f5c055b50 con 0x7f9f6815df40 2024-04-25T01:08:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.805+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9f680b2e30 con 0x7f9f6815df40 2024-04-25T01:08:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.805+0000 7f9f64ff9640 1 -- 172.21.15.33:0/229426574 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f9f5c055450 con 0x7f9f6815df40 2024-04-25T01:08:49.800 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:08:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9f540678c0 msgr2=0x7f9f54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9f540678c0 0x7f9f54069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f580af000 tx=0x7f9f58008040 comp rx=0 tx=0).stop 2024-04-25T01:08:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f6815df40 msgr2=0x7f9f68158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f6815df40 0x7f9f68158430 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00c370 tx=0x7f9f5c00c830 comp rx=0 tx=0).stop 2024-04-25T01:08:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 shutdown_connections 2024-04-25T01:08:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9f540678c0 0x7f9f54069d80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9f680c4af0 0x7f9f68144050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f6815df40 0x7f9f68158430 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 --2- 172.21.15.33:0/229426574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f6815a3a0 0x7f9f681f7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 >> 172.21.15.33:0/229426574 conn(0x7f9f680b1810 msgr2=0x7f9f680b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 shutdown_connections 2024-04-25T01:08:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.809+0000 7f9f6e425640 1 -- 172.21.15.33:0/229426574 wait complete. 2024-04-25T01:08:49.817 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:08:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26fcb6640 1 Processor -- start 2024-04-25T01:08:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26fcb6640 1 -- start start 2024-04-25T01:08:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26fcb6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26815abb0 0x7ff26806e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26fcb6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff26806ec50 0x7ff268071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26fcb6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26815af90 con 0x7ff2680afb40 2024-04-25T01:08:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26fcb6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff26806fd00 con 0x7ff26806ec50 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26fcb6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff26806f0f0 con 0x7ff26815abb0 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26d251640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff26806ec50 0x7ff268071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26d251640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff26806ec50 0x7ff268071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59390/0 (socket says 172.21.15.33:59390) 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26d251640 1 -- 172.21.15.33:0/3859101813 learned_addr learned my addr 172.21.15.33:0/3859101813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26da52640 1 --2- 172.21.15.33:0/3859101813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26815abb0 0x7ff26806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26d251640 1 -- 172.21.15.33:0/3859101813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26815abb0 msgr2=0x7ff26806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26d251640 1 --2- 172.21.15.33:0/3859101813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26815abb0 0x7ff26806e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26d251640 1 -- 172.21.15.33:0/3859101813 >> v1:172.21.15.138:6789/0 conn(0x7ff2680afb40 legacy=0x7ff2680add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.941+0000 7ff26d251640 1 -- 172.21.15.33:0/3859101813 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26806f3d0 con 0x7ff26806ec50 2024-04-25T01:08:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26d251640 1 --2- 172.21.15.33:0/3859101813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26806ec50 0x7ff268071e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d150 tx=0x7ff25c01ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3da252fde3d1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26ca50640 1 -- 172.21.15.33:0/3859101813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00b2d0 con 0x7ff26806ec50 2024-04-25T01:08:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26ca50640 1 -- 172.21.15.33:0/3859101813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff25c00b470 con 0x7ff26806ec50 2024-04-25T01:08:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26ca50640 1 -- 172.21.15.33:0/3859101813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0a7c00 con 0x7ff26806ec50 2024-04-25T01:08:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3859101813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26806ec50 msgr2=0x7ff268071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3859101813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26806ec50 0x7ff268071e30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d150 tx=0x7ff25c01ec00 comp rx=0 tx=0).stop 2024-04-25T01:08:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3859101813 shutdown_connections 2024-04-25T01:08:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3859101813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26806ec50 0x7ff268071e30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3859101813 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26815abb0 0x7ff26806e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3859101813 >> 172.21.15.33:0/3859101813 conn(0x7ff2680b28c0 msgr2=0x7ff2680b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3859101813 shutdown_connections 2024-04-25T01:08:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3859101813 wait complete. 2024-04-25T01:08:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 Processor -- start 2024-04-25T01:08:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- start start 2024-04-25T01:08:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26806ec50 0x7ff2681488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2680afb40 0x7ff268151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26815abb0 0x7ff26814fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff26806b5e0 con 0x7ff26815abb0 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff26806b460 con 0x7ff2680afb40 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26fcb6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff26806b760 con 0x7ff26806ec50 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26815abb0 0x7ff26814fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26815abb0 0x7ff26814fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59404/0 (socket says 172.21.15.33:59404) 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 -- 172.21.15.33:0/3401783241 learned_addr learned my addr 172.21.15.33:0/3401783241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26d251640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2680afb40 0x7ff268151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26da52640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26806ec50 0x7ff2681488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 -- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26806ec50 msgr2=0x7ff2681488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26806ec50 0x7ff2681488d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 -- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2680afb40 msgr2=0x7ff268151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2680afb40 0x7ff268151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 -- 172.21.15.33:0/3401783241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff268148e10 con 0x7ff26815abb0 2024-04-25T01:08:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26d251640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2680afb40 0x7ff268151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.945+0000 7ff26e253640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26815abb0 0x7ff26814fcc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c460 tx=0x7ff25800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff2667fc640 1 -- 172.21.15.33:0/3401783241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258016020 con 0x7ff26815abb0 2024-04-25T01:08:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff2667fc640 1 -- 172.21.15.33:0/3401783241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff258005220 con 0x7ff26815abb0 2024-04-25T01:08:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff268150760 con 0x7ff26815abb0 2024-04-25T01:08:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff2667fc640 1 -- 172.21.15.33:0/3401783241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff258007030 con 0x7ff26815abb0 2024-04-25T01:08:49.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff268150d20 con 0x7ff26815abb0 2024-04-25T01:08:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff268069040 con 0x7ff26815abb0 2024-04-25T01:08:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff2667fc640 1 -- 172.21.15.33:0/3401783241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2580071d0 con 0x7ff26815abb0 2024-04-25T01:08:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff2667fc640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2540678c0 0x7ff254069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff2667fc640 1 -- 172.21.15.33:0/3401783241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff2580858b0 con 0x7ff26815abb0 2024-04-25T01:08:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff26da52640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2540678c0 0x7ff254069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:49.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.949+0000 7ff26da52640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2540678c0 0x7ff254069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff26806eb00 tx=0x7ff260008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:49.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:49.953+0000 7ff2667fc640 1 -- 172.21.15.33:0/3401783241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff258020020 con 0x7ff26815abb0 2024-04-25T01:08:50.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.213+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff2680b2d40 con 0x7ff26815abb0 2024-04-25T01:08:50.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.213+0000 7ff2667fc640 1 -- 172.21.15.33:0/3401783241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7ff2580074c0 con 0x7ff26815abb0 2024-04-25T01:08:50.206 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:50.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2540678c0 msgr2=0x7ff254069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2540678c0 0x7ff254069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff26806eb00 tx=0x7ff260008040 comp rx=0 tx=0).stop 2024-04-25T01:08:50.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26815abb0 msgr2=0x7ff26814fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26815abb0 0x7ff26814fcc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff25800c460 tx=0x7ff25800c920 comp rx=0 tx=0).stop 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 shutdown_connections 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2540678c0 0x7ff254069d80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff26815abb0 0x7ff26814fcc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2680afb40 0x7ff268151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 --2- 172.21.15.33:0/3401783241 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff26806ec50 0x7ff2681488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 >> 172.21.15.33:0/3401783241 conn(0x7ff2680b28c0 msgr2=0x7ff2680b0730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 shutdown_connections 2024-04-25T01:08:50.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.217+0000 7ff26fcb6640 1 -- 172.21.15.33:0/3401783241 wait complete. 2024-04-25T01:08:50.224 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:08:50.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce14c4c640 1 Processor -- start 2024-04-25T01:08:50.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce14c4c640 1 -- start start 2024-04-25T01:08:50.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce14c4c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fce100ce300 0x7fce100cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce14c4c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce100cc840 con 0x7fce1015be70 2024-04-25T01:08:50.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce14c4c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce100ccb20 con 0x7fce1006c930 2024-04-25T01:08:50.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce14c4c640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fce100ce6e0 con 0x7fce100ce300 2024-04-25T01:08:50.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0e59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fce1015be70 0x7fce1006c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51268/0 (socket says 172.21.15.33:51268) 2024-04-25T01:08:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0e59c640 1 -- 172.21.15.33:0/403506001 learned_addr learned my addr 172.21.15.33:0/403506001 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/403506001 >> v2:172.21.15.33:3300/0 conn(0x7fce100ce300 0x7fce100cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0d59a640 1 -- 172.21.15.33:0/403506001 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4094605197 0 0) 0x7fce100cc840 con 0x7fce1015be70 2024-04-25T01:08:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0d59a640 1 -- 172.21.15.33:0/403506001 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcdfc003530 con 0x7fce1015be70 2024-04-25T01:08:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0ed9d640 1 -- 172.21.15.33:0/403506001 >> v1:172.21.15.138:6789/0 conn(0x7fce1006c930 legacy=0x7fce1006d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0ed9d640 1 -- 172.21.15.33:0/403506001 >> v1:172.21.15.33:6789/0 conn(0x7fce1015be70 legacy=0x7fce1006c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.353+0000 7fce0ed9d640 1 -- 172.21.15.33:0/403506001 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce100cce00 con 0x7fce100ce300 2024-04-25T01:08:50.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/403506001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100ce300 0x7fce100cfae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fce00009d90 tx=0x7fce0001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c5b4838ac5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0d59a640 1 -- 172.21.15.33:0/403506001 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000b2070 con 0x7fce100ce300 2024-04-25T01:08:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0d59a640 1 -- 172.21.15.33:0/403506001 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce000a58e0 con 0x7fce100ce300 2024-04-25T01:08:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0d59a640 1 -- 172.21.15.33:0/403506001 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000a5cb0 con 0x7fce100ce300 2024-04-25T01:08:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- 172.21.15.33:0/403506001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100ce300 msgr2=0x7fce100cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 --2- 172.21.15.33:0/403506001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100ce300 0x7fce100cfae0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fce00009d90 tx=0x7fce0001c930 comp rx=0 tx=0).stop 2024-04-25T01:08:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- 172.21.15.33:0/403506001 shutdown_connections 2024-04-25T01:08:50.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 --2- 172.21.15.33:0/403506001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce100ce300 0x7fce100cfae0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- 172.21.15.33:0/403506001 >> 172.21.15.33:0/403506001 conn(0x7fce100715d0 msgr2=0x7fce100719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- 172.21.15.33:0/403506001 shutdown_connections 2024-04-25T01:08:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- 172.21.15.33:0/403506001 wait complete. 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 Processor -- start 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- start start 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce1006c930 0x7fce10149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100ce300 0x7fce100adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce1015be70 0x7fce10147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fce1006e880 con 0x7fce1015be70 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fce1006e700 con 0x7fce1006c930 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fce1006ea00 con 0x7fce100ce300 2024-04-25T01:08:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce1015be70 0x7fce10147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce1015be70 0x7fce10147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59418/0 (socket says 172.21.15.33:59418) 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 -- 172.21.15.33:0/2443039851 learned_addr learned my addr 172.21.15.33:0/2443039851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0dd9b640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100ce300 0x7fce100adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0e59c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce1006c930 0x7fce10149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 -- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100ce300 msgr2=0x7fce100adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100ce300 0x7fce100adaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 -- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce1006c930 msgr2=0x7fce10149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce1006c930 0x7fce10149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 -- 172.21.15.33:0/2443039851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1014b3c0 con 0x7fce1015be70 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce0ed9d640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce1015be70 0x7fce10147880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fce0001a560 tx=0x7fce000196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce06ffd640 1 -- 172.21.15.33:0/2443039851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000b2050 con 0x7fce1015be70 2024-04-25T01:08:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce06ffd640 1 -- 172.21.15.33:0/2443039851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce000be070 con 0x7fce1015be70 2024-04-25T01:08:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce100b4a90 con 0x7fce1015be70 2024-04-25T01:08:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce06ffd640 1 -- 172.21.15.33:0/2443039851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce000ada50 con 0x7fce1015be70 2024-04-25T01:08:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.357+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce100b5050 con 0x7fce1015be70 2024-04-25T01:08:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.361+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce1014b590 con 0x7fce1015be70 2024-04-25T01:08:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.361+0000 7fce06ffd640 1 -- 172.21.15.33:0/2443039851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce000a5760 con 0x7fce1015be70 2024-04-25T01:08:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.361+0000 7fce06ffd640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcdf00678c0 0x7fcdf0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.361+0000 7fce06ffd640 1 -- 172.21.15.33:0/2443039851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce00121220 con 0x7fce1015be70 2024-04-25T01:08:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.361+0000 7fce0e59c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcdf00678c0 0x7fcdf0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.361+0000 7fce0e59c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcdf00678c0 0x7fcdf0069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce08001210 tx=0x7fce08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.365+0000 7fce06ffd640 1 -- 172.21.15.33:0/2443039851 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fce00016520 con 0x7fce1015be70 2024-04-25T01:08:50.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fce10069040 con 0x7fce1015be70 2024-04-25T01:08:50.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce06ffd640 1 -- 172.21.15.33:0/2443039851 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fce000f1a30 con 0x7fce1015be70 2024-04-25T01:08:50.517 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:08:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcdf00678c0 msgr2=0x7fcdf0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcdf00678c0 0x7fcdf0069d80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce08001210 tx=0x7fce08009040 comp rx=0 tx=0).stop 2024-04-25T01:08:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce1015be70 msgr2=0x7fce10147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce1015be70 0x7fce10147880 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fce0001a560 tx=0x7fce000196d0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 shutdown_connections 2024-04-25T01:08:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcdf00678c0 0x7fcdf0069d80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce1015be70 0x7fce10147880 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fce100ce300 0x7fce100adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 --2- 172.21.15.33:0/2443039851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce1006c930 0x7fce10149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.525+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 >> 172.21.15.33:0/2443039851 conn(0x7fce100715d0 msgr2=0x7fce101576b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:50.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.529+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 shutdown_connections 2024-04-25T01:08:50.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.529+0000 7fce14c4c640 1 -- 172.21.15.33:0/2443039851 wait complete. 2024-04-25T01:08:50.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:08:50.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.669+0000 7f9e85869640 1 Processor -- start 2024-04-25T01:08:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.669+0000 7f9e85869640 1 -- start start 2024-04-25T01:08:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e85869640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e8006eb60 0x7f9e80160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e85869640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e800cee10 con 0x7f9e80071170 2024-04-25T01:08:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e85869640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e800cf0f0 con 0x7f9e80145660 2024-04-25T01:08:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e85869640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9e80145a40 con 0x7f9e8006eb60 2024-04-25T01:08:50.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9e80071170 0x7f9e800715c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51272/0 (socket says 172.21.15.33:51272) 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 -- 172.21.15.33:0/3028238284 learned_addr learned my addr 172.21.15.33:0/3028238284 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7e7fc640 1 --2- 172.21.15.33:0/3028238284 >> v2:172.21.15.138:3300/0 conn(0x7f9e8006eb60 0x7f9e80160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 671454031 0 0) 0x7f9e800cee10 con 0x7f9e80071170 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e68003530 con 0x7f9e80071170 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 772143620 0 0) 0x7f9e68003530 con 0x7f9e80071170 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 >> v2:172.21.15.138:3300/0 conn(0x7f9e8006eb60 msgr2=0x7f9e80160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 --2- 172.21.15.33:0/3028238284 >> v2:172.21.15.138:3300/0 conn(0x7f9e8006eb60 0x7f9e80160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 >> v1:172.21.15.138:6789/0 conn(0x7f9e80145660 legacy=0x7f9e8015f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e800cf3d0 con 0x7f9e80071170 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f9e78001a00 con 0x7f9e80071170 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 >> v1:172.21.15.33:6789/0 conn(0x7f9e80071170 legacy=0x7f9e800715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e68003a10 0x7f9e68003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e68004810 0x7f9e68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680051d0 0x7f9e6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6800c0a0 con 0x7f9e680051d0 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e6800c250 con 0x7f9e68003a10 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e6800c450 con 0x7f9e68004810 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680051d0 0x7f9e6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7effd640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e68003a10 0x7f9e68003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 -- 172.21.15.33:0/3028238284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e68004810 msgr2=0x7f9e68004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e68004810 0x7f9e68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 -- 172.21.15.33:0/3028238284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e68003a10 msgr2=0x7f9e68003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e68003a10 0x7f9e68003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.673+0000 7f9e7f7fe640 1 -- 172.21.15.33:0/3028238284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e68003530 con 0x7f9e680051d0 2024-04-25T01:08:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7f7fe640 1 -- 172.21.15.33:0/3028238284 reap_dead start 2024-04-25T01:08:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7f7fe640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680051d0 0x7f9e6800b9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800a7b0 tx=0x7f9e7801d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abe7fb3366a2179 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7800a3d0 con 0x7f9e680051d0 2024-04-25T01:08:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e780ac660 con 0x7f9e680051d0 2024-04-25T01:08:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7dffb640 1 -- 172.21.15.33:0/3028238284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780aca50 con 0x7f9e680051d0 2024-04-25T01:08:50.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- 172.21.15.33:0/3028238284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680051d0 msgr2=0x7f9e6800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680051d0 0x7f9e6800b9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800a7b0 tx=0x7f9e7801d2e0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- 172.21.15.33:0/3028238284 shutdown_connections 2024-04-25T01:08:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 --2- 172.21.15.33:0/3028238284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e680051d0 0x7f9e6800b9d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- 172.21.15.33:0/3028238284 >> 172.21.15.33:0/3028238284 conn(0x7f9e800b1810 msgr2=0x7f9e800b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- 172.21.15.33:0/3028238284 shutdown_connections 2024-04-25T01:08:50.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- 172.21.15.33:0/3028238284 wait complete. 2024-04-25T01:08:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 Processor -- start 2024-04-25T01:08:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- start start 2024-04-25T01:08:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680051d0 0x7f9e8015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80155a60 0x7f9e8015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e8015c960 0x7f9e800b4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80070ce0 con 0x7f9e80155a60 2024-04-25T01:08:50.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800cec70 con 0x7f9e680051d0 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9e8006efa0 con 0x7f9e8015c960 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80155a60 0x7f9e8015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80155a60 0x7f9e8015c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59436/0 (socket says 172.21.15.33:59436) 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 -- 172.21.15.33:0/3766565822 learned_addr learned my addr 172.21.15.33:0/3766565822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7effd640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680051d0 0x7f9e8015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7f7fe640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e8015c960 0x7f9e800b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 -- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e8015c960 msgr2=0x7f9e800b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e8015c960 0x7f9e800b4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 -- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680051d0 msgr2=0x7f9e8015b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680051d0 0x7f9e8015b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 -- 172.21.15.33:0/3766565822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e800b35d0 con 0x7f9e80155a60 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7effd640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680051d0 0x7f9e8015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e7e7fc640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80155a60 0x7f9e8015c420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400e820 tx=0x7f9e7400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e67fff640 1 -- 172.21.15.33:0/3766565822 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e74017070 con 0x7f9e80155a60 2024-04-25T01:08:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e67fff640 1 -- 172.21.15.33:0/3766565822 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e7400c730 con 0x7f9e80155a60 2024-04-25T01:08:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e67fff640 1 -- 172.21.15.33:0/3766565822 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400cb00 con 0x7f9e80155a60 2024-04-25T01:08:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8015b6b0 con 0x7f9e80155a60 2024-04-25T01:08:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.677+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e800b3b50 con 0x7f9e80155a60 2024-04-25T01:08:50.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.681+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e80069040 con 0x7f9e80155a60 2024-04-25T01:08:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.681+0000 7f9e67fff640 1 -- 172.21.15.33:0/3766565822 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9e7401a070 con 0x7f9e80155a60 2024-04-25T01:08:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.681+0000 7f9e67fff640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e6c0678f0 0x7f9e6c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.681+0000 7f9e67fff640 1 -- 172.21.15.33:0/3766565822 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9e74085cc0 con 0x7f9e80155a60 2024-04-25T01:08:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.681+0000 7f9e7effd640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e6c0678f0 0x7f9e6c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.681+0000 7f9e7effd640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e6c0678f0 0x7f9e6c069db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e70001660 tx=0x7f9e70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.685+0000 7f9e67fff640 1 -- 172.21.15.33:0/3766565822 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9e74056550 con 0x7f9e80155a60 2024-04-25T01:08:50.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.837+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9e80072ac0 con 0x7f9e80155a60 2024-04-25T01:08:50.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.837+0000 7f9e67fff640 1 -- 172.21.15.33:0/3766565822 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f9e74055e50 con 0x7f9e80155a60 2024-04-25T01:08:50.831 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e6c0678f0 msgr2=0x7f9e6c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e6c0678f0 0x7f9e6c069db0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9e70001660 tx=0x7f9e70009040 comp rx=0 tx=0).stop 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80155a60 msgr2=0x7f9e8015c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80155a60 0x7f9e8015c420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400e820 tx=0x7f9e7400ece0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 shutdown_connections 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9e6c0678f0 0x7f9e6c069db0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9e8015c960 0x7f9e800b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80155a60 0x7f9e8015c420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 --2- 172.21.15.33:0/3766565822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e680051d0 0x7f9e8015b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 >> 172.21.15.33:0/3766565822 conn(0x7f9e800b1810 msgr2=0x7f9e80071580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 shutdown_connections 2024-04-25T01:08:50.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.841+0000 7f9e85869640 1 -- 172.21.15.33:0/3766565822 wait complete. 2024-04-25T01:08:50.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7f06a640 1 Processor -- start 2024-04-25T01:08:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7f06a640 1 -- start start 2024-04-25T01:08:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7f06a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7815c2d0 0x7fbc7815c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7f06a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc780d0a00 con 0x7fbc7806b210 2024-04-25T01:08:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7f06a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc7815cd90 con 0x7fbc780d0620 2024-04-25T01:08:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7f06a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7806e870 con 0x7fbc7815c2d0 2024-04-25T01:08:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbc7806b210 0x7fbc7815d490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51282/0 (socket says 172.21.15.33:51282) 2024-04-25T01:08:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 -- 172.21.15.33:0/226298351 learned_addr learned my addr 172.21.15.33:0/226298351 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:50.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc77fff640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7815c2d0 0x7fbc7815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429094865 0 0) 0x7fbc780d0a00 con 0x7fbc7806b210 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc68003530 con 0x7fbc7806b210 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1320769777 0 0) 0x7fbc68003530 con 0x7fbc7806b210 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7815c2d0 msgr2=0x7fbc7815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7815c2d0 0x7fbc7815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 >> v1:172.21.15.138:6789/0 conn(0x7fbc780d0620 legacy=0x7fbc7806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7815d070 con 0x7fbc7806b210 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fbc70001a00 con 0x7fbc7806b210 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 >> v1:172.21.15.33:6789/0 conn(0x7fbc7806b210 legacy=0x7fbc7815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc680039f0 0x7fbc68003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc68004810 0x7fbc68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7806e870 con 0x7fbc68005220 2024-04-25T01:08:50.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6800c120 con 0x7fbc68004810 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6800c320 con 0x7fbc680039f0 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc77fff640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc7815c2d0 0x7fbc7815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7ce06640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc680039f0 0x7fbc68003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc77fff640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc68004810 0x7fbc68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 -- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc680039f0 msgr2=0x7fbc68003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc680039f0 0x7fbc68003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 -- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc68004810 msgr2=0x7fbc68004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc68004810 0x7fbc68004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc7d607640 1 -- 172.21.15.33:0/226298351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc68003530 con 0x7fbc68005220 2024-04-25T01:08:50.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.969+0000 7fbc77fff640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc68004810 0x7fbc68004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:50.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7d607640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc6800b9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000a860 tx=0x7fbc7001fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e45cd6ff84e1314 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000a480 con 0x7fbc68005220 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc700aa710 con 0x7fbc68005220 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc777fe640 1 -- 172.21.15.33:0/226298351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc700aab60 con 0x7fbc68005220 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 -- 172.21.15.33:0/226298351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 msgr2=0x7fbc6800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 --2- 172.21.15.33:0/226298351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc6800b9d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000a860 tx=0x7fbc7001fae0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7d607640 1 -- 172.21.15.33:0/226298351 reap_dead start 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 -- 172.21.15.33:0/226298351 shutdown_connections 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 -- 172.21.15.33:0/226298351 >> 172.21.15.33:0/226298351 conn(0x7fbc780b29c0 msgr2=0x7fbc780b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 -- 172.21.15.33:0/226298351 shutdown_connections 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 -- 172.21.15.33:0/226298351 wait complete. 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 Processor -- start 2024-04-25T01:08:50.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.973+0000 7fbc7f06a640 1 -- start start 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc781e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc781e7670 0x7fbc781e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc781e60c0 0x7fbc780cef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc780ae070 con 0x7fbc68005220 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc780adef0 con 0x7fbc781e7670 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7815c730 con 0x7fbc781e60c0 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc781e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc781e7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59456/0 (socket says 172.21.15.33:59456) 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 -- 172.21.15.33:0/3525165522 learned_addr learned my addr 172.21.15.33:0/3525165522 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc77fff640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc781e7670 0x7fbc781e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7d607640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc781e60c0 0x7fbc780cef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 -- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc781e60c0 msgr2=0x7fbc780cef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc781e60c0 0x7fbc780cef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 -- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc781e7670 msgr2=0x7fbc781e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc781e7670 0x7fbc781e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 -- 172.21.15.33:0/3525165522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc781e65d0 con 0x7fbc68005220 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7ce06640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc781e7d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00e550 tx=0x7fbc6c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc757fa640 1 -- 172.21.15.33:0/3525165522 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0052e0 con 0x7fbc68005220 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc757fa640 1 -- 172.21.15.33:0/3525165522 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc6c018070 con 0x7fbc68005220 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc757fa640 1 -- 172.21.15.33:0/3525165522 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c003820 con 0x7fbc68005220 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc781e68b0 con 0x7fbc68005220 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc781e6e70 con 0x7fbc68005220 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc78069040 con 0x7fbc68005220 2024-04-25T01:08:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc757fa640 1 -- 172.21.15.33:0/3525165522 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbc6c0039c0 con 0x7fbc68005220 2024-04-25T01:08:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc757fa640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbc5c0678c0 0x7fbc5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc757fa640 1 -- 172.21.15.33:0/3525165522 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbc6c085dd0 con 0x7fbc68005220 2024-04-25T01:08:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc77fff640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbc5c0678c0 0x7fbc5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:50.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.977+0000 7fbc77fff640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbc5c0678c0 0x7fbc5c069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc78143e30 tx=0x7fbc64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:50.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:50.981+0000 7fbc757fa640 1 -- 172.21.15.33:0/3525165522 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbc6c056540 con 0x7fbc68005220 2024-04-25T01:08:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.177+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc780b5a90 con 0x7fbc68005220 2024-04-25T01:08:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.177+0000 7fbc757fa640 1 -- 172.21.15.33:0/3525165522 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fbc6c087020 con 0x7fbc68005220 2024-04-25T01:08:51.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:51.171 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:08:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:08:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbc5c0678c0 msgr2=0x7fbc5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbc5c0678c0 0x7fbc5c069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fbc78143e30 tx=0x7fbc64008040 comp rx=0 tx=0).stop 2024-04-25T01:08:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 msgr2=0x7fbc781e7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc781e7d30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00e550 tx=0x7fbc6c00ea10 comp rx=0 tx=0).stop 2024-04-25T01:08:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 shutdown_connections 2024-04-25T01:08:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbc5c0678c0 0x7fbc5c069d80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbc781e60c0 0x7fbc780cef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc781e7670 0x7fbc781e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 --2- 172.21.15.33:0/3525165522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc68005220 0x7fbc781e7d30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 >> 172.21.15.33:0/3525165522 conn(0x7fbc780b29c0 msgr2=0x7fbc780b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 shutdown_connections 2024-04-25T01:08:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.181+0000 7fbc7f06a640 1 -- 172.21.15.33:0/3525165522 wait complete. 2024-04-25T01:08:51.190 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:08:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd80f0a640 1 Processor -- start 2024-04-25T01:08:51.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd80f0a640 1 -- start start 2024-04-25T01:08:51.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd80f0a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd7c1445e0 0x7fdd7c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd80f0a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c0722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd80f0a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd7c06e720 con 0x7fdd7c15e820 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd80f0a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7c072800 con 0x7fdd7c1445e0 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd80f0a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c06ea00 con 0x7fdd7c071e70 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7a59c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd7c1445e0 0x7fdd7c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7a59c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdd7c1445e0 0x7fdd7c06ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59464/0 (socket says 172.21.15.33:59464) 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7a59c640 1 -- 172.21.15.33:0/2477496721 learned_addr learned my addr 172.21.15.33:0/2477496721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7a59c640 1 -- 172.21.15.33:0/2477496721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 msgr2=0x7fdd7c0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7ad9d640 1 --2- 172.21.15.33:0/2477496721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c0722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7a59c640 1 --2- 172.21.15.33:0/2477496721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c0722c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7a59c640 1 -- 172.21.15.33:0/2477496721 >> v1:172.21.15.138:6789/0 conn(0x7fdd7c15e820 legacy=0x7fdd7c06f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:51.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7a59c640 1 -- 172.21.15.33:0/2477496721 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c06ece0 con 0x7fdd7c1445e0 2024-04-25T01:08:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.325+0000 7fdd7ad9d640 1 --2- 172.21.15.33:0/2477496721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:51.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd7a59c640 1 --2- 172.21.15.33:0/2477496721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c06ded0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c073270 tx=0x7fdd6c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bc57e1676b1ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd79d9b640 1 -- 172.21.15.33:0/2477496721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0b2070 con 0x7fdd7c1445e0 2024-04-25T01:08:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd79d9b640 1 -- 172.21.15.33:0/2477496721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd6c0ad640 con 0x7fdd7c1445e0 2024-04-25T01:08:51.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd79d9b640 1 -- 172.21.15.33:0/2477496721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0ada10 con 0x7fdd7c1445e0 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- 172.21.15.33:0/2477496721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c1445e0 msgr2=0x7fdd7c06ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/2477496721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c06ded0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c073270 tx=0x7fdd6c01c4e0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- 172.21.15.33:0/2477496721 shutdown_connections 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/2477496721 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c0722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/2477496721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c06ded0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- 172.21.15.33:0/2477496721 >> 172.21.15.33:0/2477496721 conn(0x7fdd7c0b2820 msgr2=0x7fdd7c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- 172.21.15.33:0/2477496721 shutdown_connections 2024-04-25T01:08:51.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- 172.21.15.33:0/2477496721 wait complete. 2024-04-25T01:08:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 Processor -- start 2024-04-25T01:08:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- start start 2024-04-25T01:08:51.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c15e820 0x7fdd7c15fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c0ce740 con 0x7fdd7c15e820 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c06e410 con 0x7fdd7c1445e0 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd80f0a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c0ce8c0 con 0x7fdd7c071e70 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd7ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c15e820 0x7fdd7c15fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd7ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c15e820 0x7fdd7c15fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59478/0 (socket says 172.21.15.33:59478) 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.329+0000 7fdd7ad9d640 1 -- 172.21.15.33:0/1759016931 learned_addr learned my addr 172.21.15.33:0/1759016931 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7a59c640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd71d9b640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7ad9d640 1 -- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 msgr2=0x7fdd7c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7ad9d640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c15b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7ad9d640 1 -- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7c1445e0 msgr2=0x7fdd7c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7ad9d640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7ad9d640 1 -- 172.21.15.33:0/1759016931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c15be50 con 0x7fdd7c15e820 2024-04-25T01:08:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7a59c640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd71d9b640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:08:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7ad9d640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c15e820 0x7fdd7c15fd10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400ce60 tx=0x7fdd7400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd737fe640 1 -- 172.21.15.33:0/1759016931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400c2e0 con 0x7fdd7c15e820 2024-04-25T01:08:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd737fe640 1 -- 172.21.15.33:0/1759016931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7400ebf0 con 0x7fdd7c15e820 2024-04-25T01:08:51.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd737fe640 1 -- 172.21.15.33:0/1759016931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400a470 con 0x7fdd7c15e820 2024-04-25T01:08:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c15d350 con 0x7fdd7c15e820 2024-04-25T01:08:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c15f230 con 0x7fdd7c15e820 2024-04-25T01:08:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7c069040 con 0x7fdd7c15e820 2024-04-25T01:08:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd737fe640 1 -- 172.21.15.33:0/1759016931 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdd7400a610 con 0x7fdd7c15e820 2024-04-25T01:08:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd737fe640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd5c067a90 0x7fdd5c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd737fe640 1 -- 172.21.15.33:0/1759016931 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdd740883f0 con 0x7fdd7c15e820 2024-04-25T01:08:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.333+0000 7fdd7a59c640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd5c067a90 0x7fdd5c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.337+0000 7fdd7a59c640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd5c067a90 0x7fdd5c069f50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0af000 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:51.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.337+0000 7fdd737fe640 1 -- 172.21.15.33:0/1759016931 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdd74058c80 con 0x7fdd7c15e820 2024-04-25T01:08:51.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.497+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd7c13de70 con 0x7fdd7c15e820 2024-04-25T01:08:51.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.501+0000 7fdd737fe640 1 -- 172.21.15.33:0/1759016931 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fdd74058580 con 0x7fdd7c15e820 2024-04-25T01:08:51.495 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:51.495 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd5c067a90 msgr2=0x7fdd5c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd5c067a90 0x7fdd5c069f50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0af000 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c15e820 msgr2=0x7fdd7c15fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c15e820 0x7fdd7c15fd10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdd7400ce60 tx=0x7fdd7400e420 comp rx=0 tx=0).stop 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 shutdown_connections 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd5c067a90 0x7fdd5c069f50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd7c15e820 0x7fdd7c15fd10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd7c1445e0 0x7fdd7c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 --2- 172.21.15.33:0/1759016931 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdd7c071e70 0x7fdd7c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 >> 172.21.15.33:0/1759016931 conn(0x7fdd7c0b2820 msgr2=0x7fdd7c13d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 shutdown_connections 2024-04-25T01:08:51.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.505+0000 7fdd80f0a640 1 -- 172.21.15.33:0/1759016931 wait complete. 2024-04-25T01:08:51.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:08:51.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.633+0000 7fbb62983640 1 Processor -- start 2024-04-25T01:08:51.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.633+0000 7fbb62983640 1 -- start start 2024-04-25T01:08:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb62983640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbb5c1444f0 0x7fbb5c0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb62983640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbb5c15e660 0x7fbb5c071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb62983640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c06dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb62983640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c0ce0c0 con 0x7fbb5c15e660 2024-04-25T01:08:51.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb62983640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbb5c1448d0 con 0x7fbb5c1444f0 2024-04-25T01:08:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb62983640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0b5b60 con 0x7fbb5c072370 2024-04-25T01:08:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbb5c15e660 0x7fbb5c071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fbb5c15e660 0x7fbb5c071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59494/0 (socket says 172.21.15.33:59494) 2024-04-25T01:08:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1477072750 learned_addr learned my addr 172.21.15.33:0/1477072750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:51.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5bfff640 1 --2- 172.21.15.33:0/1477072750 >> v2:172.21.15.138:3300/0 conn(0x7fbb5c1444f0 0x7fbb5c0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb60f20640 1 --2- 172.21.15.33:0/1477072750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c06dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1477072750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 msgr2=0x7fbb5c06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:51.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 --2- 172.21.15.33:0/1477072750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c06dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1477072750 >> v2:172.21.15.138:3300/0 conn(0x7fbb5c1444f0 msgr2=0x7fbb5c0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:51.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 --2- 172.21.15.33:0/1477072750 >> v2:172.21.15.138:3300/0 conn(0x7fbb5c1444f0 0x7fbb5c0b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1477072750 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c15ec20 con 0x7fbb5c15e660 2024-04-25T01:08:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5b7fe640 1 --2- 172.21.15.33:0/1477072750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c15e660 0x7fbb5c071e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c06ec60 tx=0x7fbb5001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dd8e5aefe60f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5affd640 1 -- 172.21.15.33:0/1477072750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb500b0070 con 0x7fbb5c15e660 2024-04-25T01:08:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5affd640 1 -- 172.21.15.33:0/1477072750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb500ab600 con 0x7fbb5c15e660 2024-04-25T01:08:51.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.637+0000 7fbb5affd640 1 -- 172.21.15.33:0/1477072750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb500ab9d0 con 0x7fbb5c15e660 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- 172.21.15.33:0/1477072750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c15e660 msgr2=0x7fbb5c071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 --2- 172.21.15.33:0/1477072750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c15e660 0x7fbb5c071e30 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c06ec60 tx=0x7fbb5001c1d0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- 172.21.15.33:0/1477072750 shutdown_connections 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 --2- 172.21.15.33:0/1477072750 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c06dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 --2- 172.21.15.33:0/1477072750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c15e660 0x7fbb5c071e30 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 --2- 172.21.15.33:0/1477072750 >> v2:172.21.15.138:3300/0 conn(0x7fbb5c1444f0 0x7fbb5c0b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- 172.21.15.33:0/1477072750 >> 172.21.15.33:0/1477072750 conn(0x7fbb5c0b1810 msgr2=0x7fbb5c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- 172.21.15.33:0/1477072750 shutdown_connections 2024-04-25T01:08:51.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- 172.21.15.33:0/1477072750 wait complete. 2024-04-25T01:08:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 Processor -- start 2024-04-25T01:08:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- start start 2024-04-25T01:08:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c14d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c1444f0 0x7fbb5c15f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c15e660 0x7fbb5c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c0b3620 con 0x7fbb5c1444f0 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbb5c072780 con 0x7fbb5c15e660 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbb5c0b37a0 con 0x7fbb5c072370 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c1444f0 0x7fbb5c15f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c1444f0 0x7fbb5c15f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59502/0 (socket says 172.21.15.33:59502) 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1177511646 learned_addr learned my addr 172.21.15.33:0/1177511646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5bfff640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c14d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb60f20640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c15e660 0x7fbb5c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 msgr2=0x7fbb5c14d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c14d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c15e660 msgr2=0x7fbb5c15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c15e660 0x7fbb5c15a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 -- 172.21.15.33:0/1177511646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5c14ad20 con 0x7fbb5c1444f0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb5b7fe640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c1444f0 0x7fbb5c15f7b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb500ae040 tx=0x7fbb50018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb58ff9640 1 -- 172.21.15.33:0/1177511646 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb500ba020 con 0x7fbb5c1444f0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb58ff9640 1 -- 172.21.15.33:0/1177511646 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb500b6620 con 0x7fbb5c1444f0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5c15c1f0 con 0x7fbb5c1444f0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb58ff9640 1 -- 172.21.15.33:0/1177511646 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb500b0040 con 0x7fbb5c1444f0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.641+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb5c15bf10 con 0x7fbb5c1444f0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.645+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb5c069040 con 0x7fbb5c1444f0 2024-04-25T01:08:51.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.645+0000 7fbb58ff9640 1 -- 172.21.15.33:0/1177511646 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbb500bd020 con 0x7fbb5c1444f0 2024-04-25T01:08:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.645+0000 7fbb58ff9640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbb480678f0 0x7fbb48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.645+0000 7fbb58ff9640 1 -- 172.21.15.33:0/1177511646 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb5011fe30 con 0x7fbb5c1444f0 2024-04-25T01:08:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.645+0000 7fbb5bfff640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbb480678f0 0x7fbb48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:51.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.645+0000 7fbb5bfff640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbb480678f0 0x7fbb48069db0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c1659c0 tx=0x7fbb4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:51.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.649+0000 7fbb58ff9640 1 -- 172.21.15.33:0/1177511646 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbb50016110 con 0x7fbb5c1444f0 2024-04-25T01:08:51.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.837+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb5c0b2f20 con 0x7fbb5c1444f0 2024-04-25T01:08:51.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.837+0000 7fbb58ff9640 1 -- 172.21.15.33:0/1177511646 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fbb500f0630 con 0x7fbb5c1444f0 2024-04-25T01:08:51.831 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:08:51.831 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:08:51.832 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:08:51.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbb480678f0 msgr2=0x7fbb48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbb480678f0 0x7fbb48069db0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c1659c0 tx=0x7fbb4c008040 comp rx=0 tx=0).stop 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c1444f0 msgr2=0x7fbb5c15f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c1444f0 0x7fbb5c15f7b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb500ae040 tx=0x7fbb50018f70 comp rx=0 tx=0).stop 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 shutdown_connections 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbb480678f0 0x7fbb48069db0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5c15e660 0x7fbb5c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb5c1444f0 0x7fbb5c15f7b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 --2- 172.21.15.33:0/1177511646 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbb5c072370 0x7fbb5c14d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 >> 172.21.15.33:0/1177511646 conn(0x7fbb5c0b1810 msgr2=0x7fbb5c071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 shutdown_connections 2024-04-25T01:08:51.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:51.841+0000 7fbb62983640 1 -- 172.21.15.33:0/1177511646 wait complete. 2024-04-25T01:08:52.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:08:23.800275+0000', 'modified': '2024-04-25T01:08:40.521343+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4248}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4248': {'gid': 4248, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6835/486086516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6834', 'nonce': 486086516}, {'type': 'v1', 'addr': '172.21.15.33:6835', 'nonce': 486086516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4248, 'qdb_cluster': [4248]} 2024-04-25T01:08:52.252 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:08:52.252 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T01:08:52.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f687e56b640 1 Processor -- start 2024-04-25T01:08:52.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f687e56b640 1 -- start start 2024-04-25T01:08:52.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f687e56b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6878072540 0x7f68781659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f687e56b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68781448d0 con 0x7f687815e660 2024-04-25T01:08:52.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f687e56b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f687815ec20 con 0x7f68781444f0 2024-04-25T01:08:52.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f687e56b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f68780ce0c0 con 0x7f6878072540 2024-04-25T01:08:52.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f68777fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f687815e660 0x7f6878071e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51286/0 (socket says 172.21.15.33:51286) 2024-04-25T01:08:52.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f68777fe640 1 -- 172.21.15.33:0/1469346502 learned_addr learned my addr 172.21.15.33:0/1469346502 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:52.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f687cb08640 1 --2- 172.21.15.33:0/1469346502 >> v2:172.21.15.138:3300/0 conn(0x7f6878072540 0x7f68781659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2908540418 0 0) 0x7f686c001120 con 0x7f687815e660 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6860005740 con 0x7f687815e660 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3440056417 0 0) 0x7f6860005740 con 0x7f687815e660 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 >> v2:172.21.15.138:3300/0 conn(0x7f6878072540 msgr2=0x7f68781659f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 --2- 172.21.15.33:0/1469346502 >> v2:172.21.15.138:3300/0 conn(0x7f6878072540 0x7f68781659f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 >> v1:172.21.15.138:6789/0 conn(0x7f68781444f0 legacy=0x7f68780b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687815ef00 con 0x7f687815e660 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.397+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f686c001850 con 0x7f687815e660 2024-04-25T01:08:52.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 >> v1:172.21.15.33:6789/0 conn(0x7f687815e660 legacy=0x7f6878071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6860005bd0 0x7f6860005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860006a20 0x7f6860006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68600073e0 0x7f686000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68780ce0c0 con 0x7f68600073e0 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f686000e380 con 0x7f6860006a20 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f686000e580 con 0x7f6860005bd0 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68600073e0 0x7f686000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6877fff640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6860005bd0 0x7f6860005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f68777fe640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860006a20 0x7f6860006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 -- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6860005bd0 msgr2=0x7f6860005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6860005bd0 0x7f6860005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 -- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860006a20 msgr2=0x7f6860006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860006a20 0x7f6860006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 -- 172.21.15.33:0/1469346502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6860005740 con 0x7f68600073e0 2024-04-25T01:08:52.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6877fff640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6860005bd0 0x7f6860005ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 -- 172.21.15.33:0/1469346502 reap_dead start 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687cb08640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68600073e0 0x7f686000dbe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f68780af750 tx=0x7f687001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6278fcd66098888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700096b0 con 0x7f68600073e0 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68700a5640 con 0x7f68600073e0 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f6876ffd640 1 -- 172.21.15.33:0/1469346502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700a5a10 con 0x7f68600073e0 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687e56b640 1 -- 172.21.15.33:0/1469346502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68600073e0 msgr2=0x7f686000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687e56b640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68600073e0 0x7f686000dbe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f68780af750 tx=0x7f687001ca50 comp rx=0 tx=0).stop 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687e56b640 1 -- 172.21.15.33:0/1469346502 shutdown_connections 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687e56b640 1 --2- 172.21.15.33:0/1469346502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68600073e0 0x7f686000dbe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687e56b640 1 -- 172.21.15.33:0/1469346502 >> 172.21.15.33:0/1469346502 conn(0x7f68780b1810 msgr2=0x7f68780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687e56b640 1 -- 172.21.15.33:0/1469346502 shutdown_connections 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.401+0000 7f687e56b640 1 -- 172.21.15.33:0/1469346502 wait complete. 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 Processor -- start 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 -- start start 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68600073e0 0x7f687815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68781f10c0 0x7f687815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68781f1540 0x7f687815cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68780b3620 con 0x7f68781f1540 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f687815f4c0 con 0x7f68600073e0 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68780b37a0 con 0x7f68781f10c0 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68781f1540 0x7f687815cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68781f1540 0x7f687815cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59520/0 (socket says 172.21.15.33:59520) 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 -- 172.21.15.33:0/3325688506 learned_addr learned my addr 172.21.15.33:0/3325688506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f68777fe640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68781f10c0 0x7f687815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f6877fff640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68600073e0 0x7f687815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 -- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68781f10c0 msgr2=0x7f687815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68781f10c0 0x7f687815c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 -- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68600073e0 msgr2=0x7f687815a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68600073e0 0x7f687815a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 -- 172.21.15.33:0/3325688506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6878141c80 con 0x7f68781f1540 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687cb08640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68781f1540 0x7f687815cf20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f687001c880 tx=0x7f6860007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f6874ff9640 1 -- 172.21.15.33:0/3325688506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700096b0 con 0x7f68781f1540 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f6874ff9640 1 -- 172.21.15.33:0/3325688506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68700ad640 con 0x7f68781f1540 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f6874ff9640 1 -- 172.21.15.33:0/3325688506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700ada10 con 0x7f68781f1540 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6878141f60 con 0x7f68781f1540 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.405+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6878142520 con 0x7f68781f1540 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6874ff9640 1 -- 172.21.15.33:0/3325688506 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f68700adbb0 con 0x7f68781f1540 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6874ff9640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f68640678c0 0x7f6864069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6874ff9640 1 -- 172.21.15.33:0/3325688506 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f687011f660 con 0x7f68781f1540 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f68640678c0 0x7f6864069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 Processor -- start 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- start start 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f68640678c0 0x7f6864069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f687813fbb0 tx=0x7f6868009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f426d6d5e0 con 0x55f426f6d350 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f426cf4c50 con 0x55f426f23fd0 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f426d695a0 con 0x55f426f6dc70 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59526/0 (socket says 172.21.15.33:59526) 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 -- 172.21.15.33:0/2274464066 learned_addr learned my addr 172.21.15.33:0/2274464066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f687cb08640 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68777fe640 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 -- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 msgr2=0x55f426f23a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f23a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 -- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 msgr2=0x55f426f243b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f243b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 -- 172.21.15.33:0/2274464066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f426cf2d10 con 0x55f426f6d350 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f6877fff640 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6d730 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6868003d90 tx=0x7f6868012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d644b9a31e82e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f685dffb640 1 -- 172.21.15.33:0/2274464066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868026070 con 0x55f426f6d350 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f685dffb640 1 -- 172.21.15.33:0/2274464066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6868005650 con 0x55f426f6d350 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f685dffb640 1 -- 172.21.15.33:0/2274464066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868013610 con 0x55f426f6d350 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- 172.21.15.33:0/2274464066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 msgr2=0x55f426f6d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6d730 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f6868003d90 tx=0x7f6868012000 comp rx=0 tx=0).stop 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- 172.21.15.33:0/2274464066 shutdown_connections 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f243b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f23a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 --2- 172.21.15.33:0/2274464066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6d730 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- 172.21.15.33:0/2274464066 >> 172.21.15.33:0/2274464066 conn(0x55f426f1bb70 msgr2=0x55f426f1df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- 172.21.15.33:0/2274464066 shutdown_connections 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.409+0000 7f68816e4000 1 -- 172.21.15.33:0/2274464066 wait complete. 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 -- start start 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 Processor -- start 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f6e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f6ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f426d27520 con 0x55f426f6d350 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f426d363c0 con 0x55f426f23fd0 2024-04-25T01:08:52.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f426d71c10 con 0x55f426f6dc70 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 10 client.0 _ll_register_callbacks cb 0x55f426f0a5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59536/0 (socket says 172.21.15.33:59536) 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 -- 172.21.15.33:0/1306021142 learned_addr learned my addr 172.21.15.33:0/1306021142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6877fff640 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f6e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f687cb08640 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f6ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 -- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 msgr2=0x55f426f6ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f6ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 -- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 msgr2=0x55f426f6e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f6e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 -- 172.21.15.33:0/1306021142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f426d057b0 con 0x55f426f6d350 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68777fe640 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6e7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f686c0155a0 tx=0x7f686c015a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a13a728386309b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-25T01:08:52.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 1 -- 172.21.15.33:0/1306021142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c010040 con 0x55f426f6d350 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 1 -- 172.21.15.33:0/1306021142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f686c017690 con 0x55f426f6d350 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6868005650 con 0x55f426f6d350 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 1 -- 172.21.15.33:0/1306021142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f686c025050 con 0x55f426f6d350 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 10 client.4848 fetch_fsmap learned FSMap version 9 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55f426cf4ea0 con 0x55f426f6d350 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 1 -- 172.21.15.33:0/1306021142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6868005650 con 0x55f426f6d350 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 1 -- 172.21.15.33:0/1306021142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f686c028020 con 0x55f426f6d350 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 10 client.4848 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 10 client.4848 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:08:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 20 client.4848 populate_metadata read hostname 'smithi033' 2024-04-25T01:08:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6f6f0 0x55f426f7ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 4 client.4848 mds_command: new command op to 4248 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:08:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 --> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f426d08ab0 con 0x55f426f6f6f0 2024-04-25T01:08:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6877fff640 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6f6f0 0x55f426f7ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6877fff640 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6f6f0 0x55f426f7ea90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3ac1a590156093 server_cookie=871368f17a524068 in_seq=0 out_seq=0 2024-04-25T01:08:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 10 client.4848 ms_handle_connect on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:52.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 1 -- 172.21.15.33:0/1306021142 <== mds.0 v2:172.21.15.33:6834/486086516 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f426d08ab0 con 0x55f426f6f6f0 2024-04-25T01:08:52.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f6856ffd640 10 client.4848 handle_command_reply: tid=0 2024-04-25T01:08:52.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.413+0000 7f68816e4000 1 client.4848 shutdown 2024-04-25T01:08:52.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 msgr2=0x55f426f6e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6e7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f686c0155a0 tx=0x7f686c015a60 comp rx=0 tx=0).stop 2024-04-25T01:08:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 shutdown_connections 2024-04-25T01:08:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6f6f0 0x55f426f7ea90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6dc70 0x55f426f6ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f6e7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f6856ffd640 0 client.4848 ms_handle_reset on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 --2- 172.21.15.33:0/1306021142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f23fd0 0x55f426f6e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 >> 172.21.15.33:0/1306021142 conn(0x55f426f1bb70 msgr2=0x55f426f24c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 shutdown_connections 2024-04-25T01:08:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 1 -- 172.21.15.33:0/1306021142 wait complete. 2024-04-25T01:08:52.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.417+0000 7f68816e4000 20 client.4848 trim_cache size 0 max 16384 2024-04-25T01:08:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 Processor -- start 2024-04-25T01:08:52.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 -- start start 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426f4ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f23fd0 0x55f426f4d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6d350 0x55f426f4d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f68781fcab0 con 0x55f426f23fd0 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68781fbe10 con 0x55f426f1bb70 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f68781fc460 con 0x55f426f6d350 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f23fd0 0x55f426f4d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f23fd0 0x55f426f4d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59538/0 (socket says 172.21.15.33:59538) 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 -- 172.21.15.33:0/2611255797 learned_addr learned my addr 172.21.15.33:0/2611255797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 -- 172.21.15.33:0/2611255797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6d350 msgr2=0x55f426f4d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6d350 0x55f426f4d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 -- 172.21.15.33:0/2611255797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 msgr2=0x55f426f4ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f6877fff640 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426f4ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426f4ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 -- 172.21.15.33:0/2611255797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f426cf5e80 con 0x55f426f23fd0 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f68777fe640 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f23fd0 0x55f426f4d410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f686c001d60 tx=0x7f686c015f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d4b80537aa208c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f685d7fa640 1 -- 172.21.15.33:0/2611255797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c017490 con 0x55f426f23fd0 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f685d7fa640 1 -- 172.21.15.33:0/2611255797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c01d070 con 0x55f426f23fd0 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.433+0000 7f685d7fa640 1 -- 172.21.15.33:0/2611255797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c020350 con 0x55f426f23fd0 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/2611255797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f23fd0 msgr2=0x55f426f4d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f23fd0 0x55f426f4d410 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f686c001d60 tx=0x7f686c015f90 comp rx=0 tx=0).stop 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/2611255797 shutdown_connections 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f6d350 0x55f426f4d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f23fd0 0x55f426f4d410 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/2611255797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426f4ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/2611255797 >> 172.21.15.33:0/2611255797 conn(0x55f426f7a0c0 msgr2=0x55f426f4c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/2611255797 shutdown_connections 2024-04-25T01:08:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/2611255797 wait complete. 2024-04-25T01:08:52.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- start start 2024-04-25T01:08:52.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 Processor -- start 2024-04-25T01:08:52.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426fd9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f23fd0 0x55f426f7fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f80020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f426f1b9f0 con 0x55f426f6d350 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f426d3a340 con 0x55f426f1bb70 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x55f426d6c8f0 con 0x55f426f23fd0 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 10 client.0 _ll_register_callbacks cb 0x7f686c001e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f80020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f80020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59542/0 (socket says 172.21.15.33:59542) 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 -- 172.21.15.33:0/3251665497 learned_addr learned my addr 172.21.15.33:0/3251665497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6877fff640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426fd9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68777fe640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f23fd0 0x55f426f7fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 -- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f23fd0 msgr2=0x55f426f7fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f23fd0 0x55f426f7fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 -- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 msgr2=0x55f426fd9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426fd9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 -- 172.21.15.33:0/3251665497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f426d3a760 con 0x55f426f6d350 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6877fff640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426fd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f687cb08640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f80020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68700ef780 tx=0x7f68700ef420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b46971821f0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 1 -- 172.21.15.33:0/3251665497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68700bd040 con 0x55f426f6d350 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 1 -- 172.21.15.33:0/3251665497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f68700eee40 con 0x55f426f6d350 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c01d070 con 0x55f426f6d350 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 1 -- 172.21.15.33:0/3251665497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f68700ef7f0 con 0x55f426f6d350 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 10 client.4854 fetch_fsmap learned FSMap version 9 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55f426dc69e0 con 0x55f426f6d350 2024-04-25T01:08:52.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 1 -- 172.21.15.33:0/3251665497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686c01d070 con 0x55f426f6d350 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 1 -- 172.21.15.33:0/3251665497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f68700f0b20 con 0x55f426f6d350 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 10 client.4854 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 10 client.4854 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 20 client.4854 populate_metadata read hostname 'smithi033' 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6dc70 0x55f426f80560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 4 client.4854 mds_command: new command op to 4248 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 --> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] -- command(tid 0: {"prefix": "status"}) -- 0x55f426d7ab20 con 0x55f426f6dc70 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6877fff640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6dc70 0x55f426f80560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6877fff640 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6dc70 0x55f426f80560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df08f76d0f38036c server_cookie=d72bf685b1df41f0 in_seq=0 out_seq=0 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 10 client.4854 ms_handle_connect on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 1 -- 172.21.15.33:0/3251665497 <== mds.0 v2:172.21.15.33:6834/486086516 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55f426d7ab20 con 0x55f426f6dc70 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 10 client.4854 handle_command_reply: tid=0 2024-04-25T01:08:52.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 client.4854 shutdown 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 msgr2=0x55f426f80020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f80020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f68700ef780 tx=0x7f68700ef420 comp rx=0 tx=0).stop 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 shutdown_connections 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:6834/486086516,v1:172.21.15.33:6835/486086516] conn(0x55f426f6dc70 0x55f426f80560 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f426f6d350 0x55f426f80020 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x55f426f23fd0 0x55f426f7fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f6856ffd640 0 client.4854 ms_handle_reset on v2:172.21.15.33:6834/486086516 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 --2- 172.21.15.33:0/3251665497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f426f1bb70 0x55f426fd9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 >> 172.21.15.33:0/3251665497 conn(0x55f426f7a0c0 msgr2=0x55f426f1df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 shutdown_connections 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 1 -- 172.21.15.33:0/3251665497 wait complete. 2024-04-25T01:08:52.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.437+0000 7f68816e4000 20 client.4854 trim_cache size 0 max 16384 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "10df718f-e6ff-4eb1-9854-6e8a751898ff", 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4248, 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 27.996533300999999, 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 9, 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 25, 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T01:08:52.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 35.079184067 2024-04-25T01:08:52.439 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:08:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f68640678c0 msgr2=0x7f6864069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f68640678c0 0x7f6864069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f687813fbb0 tx=0x7f6868009040 comp rx=0 tx=0).stop 2024-04-25T01:08:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68781f1540 msgr2=0x7f687815cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68781f1540 0x7f687815cf20 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f687001c880 tx=0x7f6860007a50 comp rx=0 tx=0).stop 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 shutdown_connections 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f68640678c0 0x7f6864069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68781f1540 0x7f687815cf20 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f68781f10c0 0x7f687815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 --2- 172.21.15.33:0/3325688506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68600073e0 0x7f687815a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 >> 172.21.15.33:0/3325688506 conn(0x7f68780b1810 msgr2=0x7f687806d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 shutdown_connections 2024-04-25T01:08:52.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.441+0000 7f687e56b640 1 -- 172.21.15.33:0/3325688506 wait complete. 2024-04-25T01:08:52.448 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T01:08:52.458 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T01:08:52.458 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T01:08:52.458 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T01:08:52.458 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T01:08:52.467 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T01:08:52.467 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T01:08:52.467 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T01:08:52.467 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:08:52.467 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:52.468 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T01:08:52.468 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:08:52.468 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T01:08:52.468 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T01:08:52.468 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:08:52.468 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:08:52.468 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T01:08:52.468 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:08:52.468 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T01:08:52.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:52.468 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:08:52.473 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T01:08:52.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:52.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T01:08:52.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:52.526 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-25T01:08:52.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:52.694 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T01:08:52.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:52.701 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T01:08:52.759 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-04-25T01:08:52.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:08:52.760 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T01:08:52.761 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:08:52.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d60304640 1 Processor -- start 2024-04-25T01:08:52.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d60304640 1 -- start start 2024-04-25T01:08:52.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d60304640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 0x7f3d580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d60304640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5815f9a0 con 0x7f3d580b5360 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d60304640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d5815fc80 con 0x7f3d58145470 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d60304640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d580b57b0 con 0x7f3d5815c160 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d5e8a1640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3d580b5360 0x7f3d580cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51292/0 (socket says 172.21.15.33:51292) 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.901+0000 7f3d5e8a1640 1 -- 172.21.15.33:0/2183406675 learned_addr learned my addr 172.21.15.33:0/2183406675 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d89f640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 0x7f3d580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964522293 0 0) 0x7f3d5815f9a0 con 0x7f3d580b5360 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d44003530 con 0x7f3d580b5360 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2423617375 0 0) 0x7f3d44003530 con 0x7f3d580b5360 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 msgr2=0x7f3d580b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 0x7f3d580b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 >> v1:172.21.15.138:6789/0 conn(0x7f3d58145470 legacy=0x7f3d580cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5815ff60 con 0x7f3d580b5360 2024-04-25T01:08:52.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f3d50001680 con 0x7f3d580b5360 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 >> v1:172.21.15.33:6789/0 conn(0x7f3d580b5360 legacy=0x7f3d580cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d440039f0 0x7f3d44003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d44004810 0x7f3d44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 0x7f3d440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d580b57b0 con 0x7f3d440039f0 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4400ba80 con 0x7f3d44004810 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4400bc80 con 0x7f3d5815c160 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d440039f0 0x7f3d44003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d89f640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d44004810 0x7f3d44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e8a1640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 0x7f3d440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e0a0640 1 -- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 msgr2=0x7f3d440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d5815c160 0x7f3d440051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e0a0640 1 -- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d44004810 msgr2=0x7f3d44004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:52.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d44004810 0x7f3d44004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e0a0640 1 -- 172.21.15.33:0/2183406675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d44003530 con 0x7f3d440039f0 2024-04-25T01:08:52.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d89f640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d44004810 0x7f3d44004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:52.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d440039f0 0x7f3d44003e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d54001e40 tx=0x7f3d5401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb74bc4cec1ca2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54001a90 con 0x7f3d440039f0 2024-04-25T01:08:52.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d54003b80 con 0x7f3d440039f0 2024-04-25T01:08:52.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5d09e640 1 -- 172.21.15.33:0/2183406675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54005080 con 0x7f3d440039f0 2024-04-25T01:08:52.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d60304640 1 -- 172.21.15.33:0/2183406675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d440039f0 msgr2=0x7f3d44003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d60304640 1 --2- 172.21.15.33:0/2183406675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d440039f0 0x7f3d44003e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3d54001e40 tx=0x7f3d5401c740 comp rx=0 tx=0).stop 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d5e8a1640 1 -- 172.21.15.33:0/2183406675 reap_dead start 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d60304640 1 -- 172.21.15.33:0/2183406675 shutdown_connections 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d60304640 1 -- 172.21.15.33:0/2183406675 >> 172.21.15.33:0/2183406675 conn(0x7f3d580b1810 msgr2=0x7f3d580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.905+0000 7f3d60304640 1 -- 172.21.15.33:0/2183406675 shutdown_connections 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 -- 172.21.15.33:0/2183406675 wait complete. 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 Processor -- start 2024-04-25T01:08:52.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 -- start start 2024-04-25T01:08:52.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d440039f0 0x7f3d580ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5815b620 0x7f3d58142d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58143280 0x7f3d5815d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5815f750 con 0x7f3d5815b620 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5815c540 con 0x7f3d440039f0 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3d5806e850 con 0x7f3d58143280 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5815b620 0x7f3d58142d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5815b620 0x7f3d58142d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59570/0 (socket says 172.21.15.33:59570) 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 -- 172.21.15.33:0/1363545945 learned_addr learned my addr 172.21.15.33:0/1363545945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5e8a1640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58143280 0x7f3d5815d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d440039f0 0x7f3d580ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 -- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58143280 msgr2=0x7f3d5815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58143280 0x7f3d5815d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 -- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d440039f0 msgr2=0x7f3d580ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d440039f0 0x7f3d580ccd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 -- 172.21.15.33:0/1363545945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d580cd260 con 0x7f3d5815b620 2024-04-25T01:08:52.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d440039f0 0x7f3d580ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d5d89f640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5815b620 0x7f3d58142d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d48010900 tx=0x7f3d48010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d4effd640 1 -- 172.21.15.33:0/1363545945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800f040 con 0x7f3d5815b620 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d4effd640 1 -- 172.21.15.33:0/1363545945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4800ec20 con 0x7f3d5815b620 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d4effd640 1 -- 172.21.15.33:0/1363545945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4801c400 con 0x7f3d5815b620 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d580aa7f0 con 0x7f3d5815b620 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.909+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d580aadb0 con 0x7f3d5815b620 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.913+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d580ad800 con 0x7f3d5815b620 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.913+0000 7f3d4effd640 1 -- 172.21.15.33:0/1363545945 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3d4801c5a0 con 0x7f3d5815b620 2024-04-25T01:08:52.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.913+0000 7f3d4effd640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d3c0678f0 0x7f3d3c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.913+0000 7f3d4effd640 1 -- 172.21.15.33:0/1363545945 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3d48086620 con 0x7f3d5815b620 2024-04-25T01:08:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.913+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d3c0678f0 0x7f3d3c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:52.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.913+0000 7f3d5e0a0640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d3c0678f0 0x7f3d3c069db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d54005000 tx=0x7f3d54001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:52.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:52.917+0000 7f3d4effd640 1 -- 172.21.15.33:0/1363545945 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3d580ad800 con 0x7f3d5815b620 2024-04-25T01:08:53.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.117+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3d58069040 con 0x7f3d5815b620 2024-04-25T01:08:53.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.117+0000 7f3d4effd640 1 -- 172.21.15.33:0/1363545945 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f3d48018030 con 0x7f3d5815b620 2024-04-25T01:08:53.109 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:08:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d3c0678f0 msgr2=0x7f3d3c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d3c0678f0 0x7f3d3c069db0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3d54005000 tx=0x7f3d54001520 comp rx=0 tx=0).stop 2024-04-25T01:08:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5815b620 msgr2=0x7f3d58142d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5815b620 0x7f3d58142d40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3d48010900 tx=0x7f3d48010dc0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 shutdown_connections 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3d3c0678f0 0x7f3d3c069db0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3d58143280 0x7f3d5815d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d5815b620 0x7f3d58142d40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 --2- 172.21.15.33:0/1363545945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d440039f0 0x7f3d580ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 >> 172.21.15.33:0/1363545945 conn(0x7f3d580b1810 msgr2=0x7f3d580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 shutdown_connections 2024-04-25T01:08:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53.121+0000 7f3d60304640 1 -- 172.21.15.33:0/1363545945 wait complete. 2024-04-25T01:08:53.129 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:08:53.130 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:08:53.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:53.130 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-25T01:08:53.136 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:08:53.136 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:53.137 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42470sec preferred_lft 42470sec 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:26:2b:e6:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T01:08:53.138 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:53.138 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:08:53.139 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:53.139 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:53.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:08:53.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-25T01:08:53.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-25T01:08:53.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:08:53.139 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:53.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:53.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:53.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:53.268 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:08:53.320 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-25T01:08:53.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:53.322 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-25T01:08:53.368 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-25T01:08:53.369 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:08:53.369 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T01:08:53.369 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T01:08:53.369 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T01:08:53.369 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:08:53.369 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:53.369 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:53.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T01:08:53.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T01:08:53.369 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T01:08:53.369 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:53.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:53.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:53.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:53.493 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:08:53.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:53.541 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-25T01:08:53.588 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:53.588 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:53.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:53.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T01:08:53.589 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:53.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:53.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:53.713 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T01:08:53.713 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:53.713 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:53.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T01:08:53.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:08:53.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T01:08:53.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T01:08:53.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T01:08:53.714 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:53.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:53.808 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:53.808 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:53.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-25T01:08:53.808 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:08:53.808 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:53.917 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T01:08:53.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:53.917 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:53.964 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:08:53.964 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:53.964 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:54.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:54.070 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:54.121 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:54.121 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:54.121 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:54.121 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:54.121 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:54.122 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:54.123 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:54.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:54.125 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:54.172 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:08:54.173 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T01:08:54.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:54.232 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:54.281 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:54.282 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:54.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:54.285 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:54.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2024-04-25T01:08:54.309+0000 7f9d030fc580 -1 init, newargv = 0x5583a9d4acb0 newargc=15 2024-04-25T01:08:54.302 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[35421]: starting ceph client 2024-04-25T01:08:54.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[35421]: starting fuse 2024-04-25T01:08:55.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:55.426 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:55.474 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:55.475 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:55.476 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:55.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:55.477 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:55.524 INFO:teuthology.orchestra.run.smithi033.stdout:45 2024-04-25T01:08:55.525 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:55.525 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2024-04-25T01:08:55.526 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-25T01:08:55.656 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.35421.asok 2024-04-25T01:08:55.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:55.669 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35421.asok 2024-04-25T01:08:55.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:55.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35421.asok status 2024-04-25T01:08:55.826 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "35421", 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4866, 2024-04-25T01:08:55.827 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4866 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2084043514 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2084043514 2024-04-25T01:08:55.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4866 192.168.0.1:0/2084043514", 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/2084043514", 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 9, 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 25, 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-25T01:08:55.829 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:08:55.841 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:08:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f434024a640 1 Processor -- start 2024-04-25T01:08:55.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f434024a640 1 -- start start 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f434024a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f433806b920 0x7f43380725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f434024a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433815e5d0 con 0x7f43380b0040 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f434024a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433815e8b0 con 0x7f43380cfc60 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f434024a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f433815eb90 con 0x7f433806b920 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433dfe6640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f43380b0040 0x7f433806b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:51294/0 (socket says 172.21.15.33:51294) 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433dfe6640 1 -- 172.21.15.33:0/1738997016 learned_addr learned my addr 172.21.15.33:0/1738997016 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433d7e5640 1 --2- 172.21.15.33:0/1738997016 >> v2:172.21.15.138:3300/0 conn(0x7f433806b920 0x7f43380725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3774190473 0 0) 0x7f433815e5d0 con 0x7f43380b0040 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4324003530 con 0x7f43380b0040 2024-04-25T01:08:55.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3925880233 0 0) 0x7f4324003530 con 0x7f43380b0040 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 >> v2:172.21.15.138:3300/0 conn(0x7f433806b920 msgr2=0x7f43380725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 --2- 172.21.15.33:0/1738997016 >> v2:172.21.15.138:3300/0 conn(0x7f433806b920 0x7f43380725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 >> v1:172.21.15.138:6789/0 conn(0x7f43380cfc60 legacy=0x7f4338161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433815ee70 con 0x7f43380b0040 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f4334001680 con 0x7f43380b0040 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 >> v1:172.21.15.33:6789/0 conn(0x7f43380b0040 legacy=0x7f433806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43240039c0 0x7f4324003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f4324004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43240051d0 0x7f432400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f433815eb90 con 0x7f4324004810 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f432400c170 con 0x7f43240051d0 2024-04-25T01:08:55.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f432400c370 con 0x7f43240039c0 2024-04-25T01:08:55.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433d7e5640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f4324004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433dfe6640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43240039c0 0x7f4324003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433e7e7640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43240051d0 0x7f432400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433d7e5640 1 -- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43240039c0 msgr2=0x7f4324003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:55.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433d7e5640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43240039c0 0x7f4324003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433d7e5640 1 -- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43240051d0 msgr2=0x7f432400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433d7e5640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43240051d0 0x7f432400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433d7e5640 1 -- 172.21.15.33:0/1738997016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4324003530 con 0x7f4324004810 2024-04-25T01:08:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.977+0000 7f433e7e7640 1 -- 172.21.15.33:0/1738997016 reap_dead start 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f433d7e5640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f4324004c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4328009ab0 tx=0x7f432801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9330d8748b36a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280096b0 con 0x7f4324004810 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43280a6430 con 0x7f4324004810 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f433cfe4640 1 -- 172.21.15.33:0/1738997016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280a6820 con 0x7f4324004810 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f434024a640 1 -- 172.21.15.33:0/1738997016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 msgr2=0x7f4324004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f434024a640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f4324004c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4328009ab0 tx=0x7f432801ca50 comp rx=0 tx=0).stop 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f434024a640 1 -- 172.21.15.33:0/1738997016 shutdown_connections 2024-04-25T01:08:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f434024a640 1 --2- 172.21.15.33:0/1738997016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f4324004c90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f434024a640 1 -- 172.21.15.33:0/1738997016 >> 172.21.15.33:0/1738997016 conn(0x7f43380b2820 msgr2=0x7f43380b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f434024a640 1 -- 172.21.15.33:0/1738997016 shutdown_connections 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.981+0000 7f434024a640 1 -- 172.21.15.33:0/1738997016 wait complete. 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 Processor -- start 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 -- start start 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f43381533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43380aeaf0 0x7f43381f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43381f7ba0 0x7f43381eb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4338161f40 con 0x7f4324004810 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4338161dc0 con 0x7f43381f7ba0 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f43381620c0 con 0x7f43380aeaf0 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f43381533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f43381533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59584/0 (socket says 172.21.15.33:59584) 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 -- 172.21.15.33:0/2131177470 learned_addr learned my addr 172.21.15.33:0/2131177470 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433d7e5640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43380aeaf0 0x7f43381f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433e7e7640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43381f7ba0 0x7f43381eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 -- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43380aeaf0 msgr2=0x7f43381f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43380aeaf0 0x7f43381f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 -- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43381f7ba0 msgr2=0x7f43381eb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43381f7ba0 0x7f43381eb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 -- 172.21.15.33:0/2131177470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43380aef80 con 0x7f4324004810 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f433dfe6640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f43381533d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f433400e810 tx=0x7f433400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f432e7fc640 1 -- 172.21.15.33:0/2131177470 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400c8d0 con 0x7f4324004810 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f432e7fc640 1 -- 172.21.15.33:0/2131177470 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4334018070 con 0x7f4324004810 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43380af260 con 0x7f4324004810 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f432e7fc640 1 -- 172.21.15.33:0/2131177470 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400cc90 con 0x7f4324004810 2024-04-25T01:08:55.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43380af820 con 0x7f4324004810 2024-04-25T01:08:55.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4338069040 con 0x7f4324004810 2024-04-25T01:08:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.985+0000 7f432e7fc640 1 -- 172.21.15.33:0/2131177470 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f433401d460 con 0x7f4324004810 2024-04-25T01:08:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.989+0000 7f432e7fc640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f431c0678c0 0x7f431c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.989+0000 7f432e7fc640 1 -- 172.21.15.33:0/2131177470 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4334085f80 con 0x7f4324004810 2024-04-25T01:08:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.989+0000 7f433d7e5640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f431c0678c0 0x7f431c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:55.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.989+0000 7f433d7e5640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f431c0678c0 0x7f431c069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43280ad000 tx=0x7f43280b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:55.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:55.993+0000 7f432e7fc640 1 -- 172.21.15.33:0/2131177470 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4334056090 con 0x7f4324004810 2024-04-25T01:08:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.189+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4338162790 con 0x7f4324004810 2024-04-25T01:08:56.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.189+0000 7f432e7fc640 1 -- 172.21.15.33:0/2131177470 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f433405a390 con 0x7f4324004810 2024-04-25T01:08:56.182 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:08:56.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f431c0678c0 msgr2=0x7f431c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:56.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f431c0678c0 0x7f431c069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f43280ad000 tx=0x7f43280b2040 comp rx=0 tx=0).stop 2024-04-25T01:08:56.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 msgr2=0x7f43381533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:56.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f43381533d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f433400e810 tx=0x7f433400ecd0 comp rx=0 tx=0).stop 2024-04-25T01:08:56.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 shutdown_connections 2024-04-25T01:08:56.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f431c0678c0 0x7f431c069d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:56.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f43381f7ba0 0x7f43381eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:56.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f43380aeaf0 0x7f43381f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:56.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 --2- 172.21.15.33:0/2131177470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4324004810 0x7f43381533d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:56.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 >> 172.21.15.33:0/2131177470 conn(0x7f43380b2820 msgr2=0x7f4338160d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:56.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 shutdown_connections 2024-04-25T01:08:56.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56.193+0000 7f434024a640 1 -- 172.21.15.33:0/2131177470 wait complete. 2024-04-25T01:08:56.200 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:08:56.200 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:08:56.200 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-25T01:08:56.201 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T01:08:56.201 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T01:08:56.201 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:08:56.201 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:08:56.201 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:08:56.201 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T01:08:56.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:56.201 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:56.207 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42466sec preferred_lft 42466sec 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:56.208 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:26:2b:e6:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T01:08:56.209 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:56.210 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:08:56.258 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-25T01:08:56.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:56.259 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-25T01:08:56.309 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-25T01:08:56.309 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:08:56.310 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T01:08:56.310 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T01:08:56.310 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T01:08:56.310 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:08:56.310 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:56.310 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:56.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T01:08:56.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T01:08:56.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T01:08:56.310 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:56.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:56.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:56.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:56.434 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:08:56.440 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T01:08:56.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:56.441 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-25T01:08:56.488 INFO:teuthology.orchestra.run.smithi033.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T01:08:56.489 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:56.489 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:56.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:08:56.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T01:08:56.489 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:56.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:56.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:56.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 76:2c:5e:c8:96:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T01:08:56.621 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::742c:5eff:fec8:963e/64 scope link 2024-04-25T01:08:56.622 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:56.623 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 76:2c:5e:c8:96:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T01:08:56.674 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T01:08:56.675 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.675 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::742c:5eff:fec8:963e/64 scope link 2024-04-25T01:08:56.675 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:56.676 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T01:08:56.676 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:56.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:56.823 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:56.823 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:08:56.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.1 up 2024-04-25T01:08:56.823 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T01:08:56.823 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:08:56.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:56.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:56.885 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T01:08:56.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:56.885 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T01:08:56.932 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T01:08:56.932 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:56.932 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T01:08:56.980 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:57.036 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:57.085 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:57.085 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:57.085 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:57.088 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:57.088 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:57.088 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:08:57.088 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:08:57.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:57.089 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:57.136 INFO:teuthology.orchestra.run.smithi033.stdout:45 2024-04-25T01:08:57.137 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T01:08:57.137 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T01:08:57.181 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:57.197 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:57.245 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:57.246 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:08:57.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:57.247 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:57.266 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi033.stderr:2024-04-25T01:08:57.273+0000 7fa2d1035580 -1 init, newargv = 0x5582c5925780 newargc=15 2024-04-25T01:08:57.266 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi033.stderr:ceph-fuse[35566]: starting ceph client 2024-04-25T01:08:57.270 INFO:teuthology.orchestra.run.smithi033.stdout:45 2024-04-25T01:08:57.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi033.stderr:ceph-fuse[35566]: starting fuse 2024-04-25T01:08:58.371 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:08:58.387 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:08:58.435 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:08:58.435 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:08:58.435 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:08:58.435 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:08:58.435 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:08:58.435 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:08:58.436 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:58.437 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:08:58.438 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T01:08:58.438 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:08:58.438 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:08:58.438 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T01:08:58.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:58.439 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:08:58.485 INFO:teuthology.orchestra.run.smithi033.stdout:45 2024-04-25T01:08:58.485 INFO:teuthology.orchestra.run.smithi033.stdout:46 2024-04-25T01:08:58.485 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T01:08:58.485 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:08:58.485 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-25T01:08:58.485 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-25T01:08:58.486 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.2")) 2024-04-25T01:08:58.487 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-25T01:08:58.613 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.2.35566.asok 2024-04-25T01:08:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:58.625 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.35566.asok 2024-04-25T01:08:58.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:58.625 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.35566.asok status 2024-04-25T01:08:58.783 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:08:58.783 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-25T01:08:58.783 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:08:58.783 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:08:58.783 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "2", 2024-04-25T01:08:58.783 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "35566", 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 4875, 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 4875 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:58.784 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.2:0", 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3916293855 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.2:0", 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3916293855 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.4875 192.168.0.2:0/3916293855", 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.2:0/3916293855", 2024-04-25T01:08:58.785 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-04-25T01:08:58.786 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 9, 2024-04-25T01:08:58.786 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 25, 2024-04-25T01:08:58.786 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:08:58.786 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-25T01:08:58.786 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-25T01:08:58.786 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:08:58.798 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:08:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.933+0000 7fefe2b3e640 1 Processor -- start 2024-04-25T01:08:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.933+0000 7fefe2b3e640 1 -- start start 2024-04-25T01:08:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe2b3e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc15c160 0x7fefdc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe2b3e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fefdc0b5360 0x7fefdc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe2b3e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefdc15f9a0 con 0x7fefdc145470 2024-04-25T01:08:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe2b3e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fefdc0b57b0 con 0x7fefdc0b5360 2024-04-25T01:08:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe2b3e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fefdc15f750 con 0x7fefdc15c160 2024-04-25T01:08:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe10db640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fefdc0b5360 0x7fefdc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe08da640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fefdc145470 0x7fefdc0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39094/0 (socket says 172.21.15.33:39094) 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe08da640 1 -- 172.21.15.33:0/1118990511 learned_addr learned my addr 172.21.15.33:0/1118990511 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefd3fff640 1 --2- 172.21.15.33:0/1118990511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc15c160 0x7fefdc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe10db640 1 -- 172.21.15.33:0/1118990511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc15c160 msgr2=0x7fefdc0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe10db640 1 --2- 172.21.15.33:0/1118990511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc15c160 0x7fefdc0b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe10db640 1 -- 172.21.15.33:0/1118990511 >> v1:172.21.15.33:6789/0 conn(0x7fefdc145470 legacy=0x7fefdc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe10db640 1 -- 172.21.15.33:0/1118990511 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc15fc80 con 0x7fefdc0b5360 2024-04-25T01:08:58.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefd37fe640 1 -- 172.21.15.33:0/1118990511 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71802249 0 0) 0x7fefdc15f9a0 con 0x7fefdc145470 2024-04-25T01:08:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefe10db640 1 --2- 172.21.15.33:0/1118990511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc0b5360 0x7fefdc0cce00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009d90 tx=0x7fefd401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b80d028fdc2dd5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefd37fe640 1 -- 172.21.15.33:0/1118990511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40b2070 con 0x7fefdc0b5360 2024-04-25T01:08:58.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefd37fe640 1 -- 172.21.15.33:0/1118990511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd40a5940 con 0x7fefdc0b5360 2024-04-25T01:08:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.937+0000 7fefd37fe640 1 -- 172.21.15.33:0/1118990511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40a5d10 con 0x7fefdc0b5360 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- 172.21.15.33:0/1118990511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc0b5360 msgr2=0x7fefdc0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/1118990511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc0b5360 0x7fefdc0cce00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fefd4009d90 tx=0x7fefd401c970 comp rx=0 tx=0).stop 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- 172.21.15.33:0/1118990511 shutdown_connections 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/1118990511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc0b5360 0x7fefdc0cce00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/1118990511 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc15c160 0x7fefdc0b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- 172.21.15.33:0/1118990511 >> 172.21.15.33:0/1118990511 conn(0x7fefdc0b1810 msgr2=0x7fefdc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- 172.21.15.33:0/1118990511 shutdown_connections 2024-04-25T01:08:58.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- 172.21.15.33:0/1118990511 wait complete. 2024-04-25T01:08:58.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 Processor -- start 2024-04-25T01:08:58.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- start start 2024-04-25T01:08:58.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc0b5360 0x7fefdc13cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:58.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc145470 0x7fefdc13d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:58.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc15c160 0x7fefdc15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc06e850 con 0x7fefdc15c160 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fefdc145850 con 0x7fefdc0b5360 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fefdc168af0 con 0x7fefdc145470 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc15c160 0x7fefdc15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc15c160 0x7fefdc15a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51276/0 (socket says 172.21.15.33:51276) 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 -- 172.21.15.33:0/3955094833 learned_addr learned my addr 172.21.15.33:0/3955094833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe08da640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc0b5360 0x7fefdc13cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefd3fff640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc145470 0x7fefdc13d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 -- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc145470 msgr2=0x7fefdc13d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc145470 0x7fefdc13d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 -- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc0b5360 msgr2=0x7fefdc13cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:58.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc0b5360 0x7fefdc13cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 -- 172.21.15.33:0/3955094833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefdc13dc10 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe10db640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc15c160 0x7fefdc15a0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fefd40b0040 tx=0x7fefd4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefd17fa640 1 -- 172.21.15.33:0/3955094833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40b2050 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefd17fa640 1 -- 172.21.15.33:0/3955094833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefd40be070 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefdc142650 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefd17fa640 1 -- 172.21.15.33:0/3955094833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefd40ada50 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.941+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefdc142c10 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.945+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefdc069040 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.945+0000 7fefd17fa640 1 -- 172.21.15.33:0/3955094833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fefd40adbf0 con 0x7fefdc15c160 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.945+0000 7fefd17fa640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fefc80678c0 0x7fefc8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:08:58.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.945+0000 7fefd17fa640 1 -- 172.21.15.33:0/3955094833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fefd4121150 con 0x7fefdc15c160 2024-04-25T01:08:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.945+0000 7fefe08da640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fefc80678c0 0x7fefc8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:08:58.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.945+0000 7fefe08da640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fefc80678c0 0x7fefc8069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000f70 tx=0x7fefd8007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:08:58.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:58.949+0000 7fefd17fa640 1 -- 172.21.15.33:0/3955094833 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fefd4016560 con 0x7fefdc15c160 2024-04-25T01:08:59.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.149+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fefdc071420 con 0x7fefdc15c160 2024-04-25T01:08:59.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.149+0000 7fefd17fa640 1 -- 172.21.15.33:0/3955094833 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fefd40f1940 con 0x7fefdc15c160 2024-04-25T01:08:59.143 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:08:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fefc80678c0 msgr2=0x7fefc8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fefc80678c0 0x7fefc8069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fefd8000f70 tx=0x7fefd8007c10 comp rx=0 tx=0).stop 2024-04-25T01:08:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc15c160 msgr2=0x7fefdc15a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:08:59.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc15c160 0x7fefdc15a0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fefd40b0040 tx=0x7fefd4019710 comp rx=0 tx=0).stop 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 shutdown_connections 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fefc80678c0 0x7fefc8069d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefdc15c160 0x7fefdc15a0a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fefdc145470 0x7fefdc13d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 --2- 172.21.15.33:0/3955094833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefdc0b5360 0x7fefdc13cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 >> 172.21.15.33:0/3955094833 conn(0x7fefdc0b1810 msgr2=0x7fefdc141260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 shutdown_connections 2024-04-25T01:08:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:08:59.153+0000 7fefe2b3e640 1 -- 172.21.15.33:0/3955094833 wait complete. 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:08:59.163 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:08:59.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:59.164 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:59.171 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42465sec preferred_lft 42465sec 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 02:42:0b:8b:bf:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T01:08:59.172 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:08:59.173 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:08:59.173 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:59.173 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:08:59.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:08:59.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-04-25T01:08:59.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-04-25T01:08:59.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:08:59.173 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:08:59.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:59.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:59.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:59.303 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:08:59.313 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T01:08:59.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:59.315 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-04-25T01:08:59.382 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-04-25T01:08:59.382 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:08:59.382 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T01:08:59.382 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:08:59.382 INFO:teuthology.orchestra.run.smithi138.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T01:08:59.382 INFO:teuthology.orchestra.run.smithi138.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T01:08:59.383 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:08:59.383 INFO:teuthology.orchestra.run.smithi138.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T01:08:59.383 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T01:08:59.383 INFO:teuthology.orchestra.run.smithi138.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T01:08:59.383 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:08:59.383 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:59.383 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:08:59.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:08:59.383 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:08:59.384 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:08:59.384 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:08:59.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:59.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:59.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:59.503 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T01:08:59.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:08:59.557 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-04-25T01:08:59.607 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:59.607 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:08:59.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:08:59.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T01:08:59.608 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:08:59.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:59.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:59.724 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T01:08:59.724 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:59.724 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:08:59.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T01:08:59.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:08:59.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T01:08:59.724 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T01:08:59.725 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T01:08:59.725 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:08:59.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:59.828 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:08:59.828 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:08:59.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-04-25T01:08:59.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:08:59.829 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:08:59.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:08:59.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:08:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:08:59.930 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T01:08:59.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:59.930 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T01:08:59.978 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:08:59.978 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:08:59.978 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:00.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:09:00.028 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:09:00.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:09:00.086 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:00.165 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:09:00.166 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:09:00.167 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:09:00.168 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:00.169 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:00.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:00.170 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:09:00.221 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:09:00.222 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T01:09:00.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:09:00.278 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:00.330 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:09:00.330 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:09:00.330 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:09:00.330 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T01:09:00.331 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:00.332 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:00.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:00.333 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:09:00.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:2024-04-25T01:09:00.340+0000 7f8d691e4580 -1 init, newargv = 0x557cafcb8bd0 newargc=15 2024-04-25T01:09:00.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[33183]: starting ceph client 2024-04-25T01:09:00.344 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[33183]: starting fuse 2024-04-25T01:09:01.443 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:09:01.459 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:09:01.508 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:09:01.509 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:09:01.510 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:09:01.510 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:09:01.510 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:09:01.510 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:09:01.510 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:01.510 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:01.510 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:01.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:01.511 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:09:01.563 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T01:09:01.564 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-04-25T01:09:01.564 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-04-25T01:09:01.565 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-04-25T01:09:01.566 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:09:01.566 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:01.566 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.1")) 2024-04-25T01:09:01.566 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-04-25T01:09:01.685 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.1.33183.asok 2024-04-25T01:09:01.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:01.697 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33183.asok 2024-04-25T01:09:01.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:01.698 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33183.asok status 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "1", 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T01:09:01.854 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "33183", 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4538, 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4538 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:09:01.855 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1616701849 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1616701849 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4538 192.168.0.1:0/1616701849", 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/1616701849", 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 9, 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 25, 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-04-25T01:09:01.856 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T01:09:01.868 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:09:02.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb4218d9640 1 Processor -- start 2024-04-25T01:09:02.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb4218d9640 1 -- start start 2024-04-25T01:09:02.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb4218d9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 0x7fb41c0b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb4218d9640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb41c06c320 0x7fb41c0b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb4218d9640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb41c0cfbc0 con 0x7fb41c0d1480 2024-04-25T01:09:02.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb4218d9640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb41c0b16c0 con 0x7fb41c06c320 2024-04-25T01:09:02.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb4218d9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb41c0d1900 con 0x7fb41c0c4bf0 2024-04-25T01:09:02.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb413fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb41c0d1480 0x7fb41c0c4fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39106/0 (socket says 172.21.15.33:39106) 2024-04-25T01:09:02.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb413fff640 1 -- 172.21.15.33:0/3511296578 learned_addr learned my addr 172.21.15.33:0/3511296578 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41b7fe640 1 --2- 172.21.15.33:0/3511296578 >> v2:172.21.15.138:3300/0 conn(0x7fb41c06c320 0x7fb41c0b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41affd640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 0x7fb41c0b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068548176 0 0) 0x7fb41c0cfbc0 con 0x7fb41c0d1480 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3fc003530 con 0x7fb41c0d1480 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2667262518 0 0) 0x7fb3fc003530 con 0x7fb41c0d1480 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 msgr2=0x7fb41c0b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 0x7fb41c0b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 >> v2:172.21.15.138:3300/0 conn(0x7fb41c06c320 msgr2=0x7fb41c0b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 --2- 172.21.15.33:0/3511296578 >> v2:172.21.15.138:3300/0 conn(0x7fb41c06c320 0x7fb41c0b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c0cfe30 con 0x7fb41c0d1480 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fb404001680 con 0x7fb41c0d1480 2024-04-25T01:09:02.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 >> v1:172.21.15.33:6789/0 conn(0x7fb41c0d1480 legacy=0x7fb41c0c4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3fc0039c0 0x7fb3fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 0x7fb3fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc004d50 0x7fb3fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0b16c0 con 0x7fb3fc0039c0 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0d1900 con 0x7fb3fc004d50 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb3fc00b9d0 con 0x7fb41c0c4bf0 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41affd640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3fc0039c0 0x7fb3fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41b7fe640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc004d50 0x7fb3fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb413fff640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 0x7fb3fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41affd640 1 -- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 msgr2=0x7fb3fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41affd640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0c4bf0 0x7fb3fc004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41affd640 1 -- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc004d50 msgr2=0x7fb3fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41affd640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc004d50 0x7fb3fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41affd640 1 -- 172.21.15.33:0/3511296578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc003530 con 0x7fb3fc0039c0 2024-04-25T01:09:02.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.013+0000 7fb41b7fe640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc004d50 0x7fb3fc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb41affd640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3fc0039c0 0x7fb3fc003de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009ae0 tx=0x7fb40c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738737081640bf0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0b2070 con 0x7fb3fc0039c0 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb40c0a5520 con 0x7fb3fc0039c0 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb41a7fc640 1 -- 172.21.15.33:0/3511296578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0096b0 con 0x7fb3fc0039c0 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 -- 172.21.15.33:0/3511296578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3fc0039c0 msgr2=0x7fb3fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 --2- 172.21.15.33:0/3511296578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb3fc0039c0 0x7fb3fc003de0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009ae0 tx=0x7fb40c01ca40 comp rx=0 tx=0).stop 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb41b7fe640 1 -- 172.21.15.33:0/3511296578 reap_dead start 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 -- 172.21.15.33:0/3511296578 shutdown_connections 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 -- 172.21.15.33:0/3511296578 >> 172.21.15.33:0/3511296578 conn(0x7fb41c0b3770 msgr2=0x7fb41c0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 -- 172.21.15.33:0/3511296578 shutdown_connections 2024-04-25T01:09:02.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 -- 172.21.15.33:0/3511296578 wait complete. 2024-04-25T01:09:02.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 Processor -- start 2024-04-25T01:09:02.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.017+0000 7fb4218d9640 1 -- start start 2024-04-25T01:09:02.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc0039c0 0x7fb41c06fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb41c0b4860 0x7fb41c0b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0b6a50 0x7fb41c0b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0b4390 con 0x7fb41c0b4860 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb41c0b4510 con 0x7fb3fc0039c0 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb41c0b2090 con 0x7fb41c0b6a50 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb41c0b4860 0x7fb41c0b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb41c0b4860 0x7fb41c0b4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51288/0 (socket says 172.21.15.33:51288) 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 -- 172.21.15.33:0/2356144000 learned_addr learned my addr 172.21.15.33:0/2356144000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb41affd640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc0039c0 0x7fb41c06fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb41b7fe640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0b6a50 0x7fb41c0b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 -- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0b6a50 msgr2=0x7fb41c0b6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0b6a50 0x7fb41c0b6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 -- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc0039c0 msgr2=0x7fb41c06fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc0039c0 0x7fb41c06fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 -- 172.21.15.33:0/2356144000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41c0b5320 con 0x7fb41c0b4860 2024-04-25T01:09:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb413fff640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb41c0b4860 0x7fb41c0b4ce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb404000f70 tx=0x7fb40400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb418ff9640 1 -- 172.21.15.33:0/2356144000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404019070 con 0x7fb41c0b4860 2024-04-25T01:09:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb418ff9640 1 -- 172.21.15.33:0/2356144000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4040072d0 con 0x7fb41c0b4860 2024-04-25T01:09:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41c0c60e0 con 0x7fb41c0b4860 2024-04-25T01:09:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb418ff9640 1 -- 172.21.15.33:0/2356144000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4040047f0 con 0x7fb41c0b4860 2024-04-25T01:09:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb41c1588d0 con 0x7fb41c0b4860 2024-04-25T01:09:02.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb41c069040 con 0x7fb41c0b4860 2024-04-25T01:09:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb418ff9640 1 -- 172.21.15.33:0/2356144000 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb40401c070 con 0x7fb41c0b4860 2024-04-25T01:09:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb418ff9640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4000678c0 0x7fb400069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb418ff9640 1 -- 172.21.15.33:0/2356144000 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb404085940 con 0x7fb41c0b4860 2024-04-25T01:09:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.021+0000 7fb41affd640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4000678c0 0x7fb400069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:02.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.025+0000 7fb41affd640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4000678c0 0x7fb400069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0af000 tx=0x7fb40c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:02.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.025+0000 7fb418ff9640 1 -- 172.21.15.33:0/2356144000 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb404056140 con 0x7fb41c0b4860 2024-04-25T01:09:02.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.221+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb41c0d1f60 con 0x7fb41c0b4860 2024-04-25T01:09:02.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb418ff9640 1 -- 172.21.15.33:0/2356144000 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fb404055a40 con 0x7fb41c0b4860 2024-04-25T01:09:02.216 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T01:09:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4000678c0 msgr2=0x7fb400069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4000678c0 0x7fb400069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0af000 tx=0x7fb40c0046d0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb41c0b4860 msgr2=0x7fb41c0b4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb41c0b4860 0x7fb41c0b4ce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb404000f70 tx=0x7fb40400dc00 comp rx=0 tx=0).stop 2024-04-25T01:09:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 shutdown_connections 2024-04-25T01:09:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4000678c0 0x7fb400069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb41c0b6a50 0x7fb41c0b6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb41c0b4860 0x7fb41c0b4ce0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 --2- 172.21.15.33:0/2356144000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3fc0039c0 0x7fb41c06fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 >> 172.21.15.33:0/2356144000 conn(0x7fb41c0b3770 msgr2=0x7fb41c070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 shutdown_connections 2024-04-25T01:09:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:02.225+0000 7fb4218d9640 1 -- 172.21.15.33:0/2356144000 wait complete. 2024-04-25T01:09:02.233 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:09:02.234 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T01:09:02.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:02.234 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-04-25T01:09:02.241 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-04-25T01:09:02.242 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42462sec preferred_lft 42462sec 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 02:42:0b:8b:bf:c1 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:02.243 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run.smithi138.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:02.244 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:09:02.300 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T01:09:02.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:02.302 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T01:09:02.356 INFO:teuthology.orchestra.run.smithi138.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T01:09:02.357 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:09:02.357 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:09:02.357 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:09:02.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:09:02.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:09:02.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:09:02.357 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:09:02.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:09:02.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:02.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:02.482 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T01:09:02.488 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T01:09:02.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:02.489 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-04-25T01:09:02.535 INFO:teuthology.orchestra.run.smithi138.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T01:09:02.535 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:09:02.535 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:09:02.535 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:09:02.535 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T01:09:02.535 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:09:02.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:09:02.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:02.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:02.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T01:09:02.660 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:09:02.660 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:09:02.660 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether fa:12:70:e3:2c:f7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::f812:70ff:fee3:2cf7/64 scope link 2024-04-25T01:09:02.661 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:02.662 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T01:09:02.718 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:09:02.718 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:09:02.718 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:09:02.718 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.718 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T01:09:02.719 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.719 INFO:teuthology.orchestra.run.smithi138.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T01:09:02.719 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether fa:12:70:e3:2c:f7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T01:09:02.719 INFO:teuthology.orchestra.run.smithi138.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T01:09:02.719 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.719 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::f812:70ff:fee3:2cf7/64 scope link 2024-04-25T01:09:02.719 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:02.720 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T01:09:02.720 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:09:02.720 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:09:02.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T01:09:02.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T01:09:02.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T01:09:02.720 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T01:09:02.721 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T01:09:02.721 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:09:02.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:09:02.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:02.861 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:09:02.861 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:09:02.862 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.1 up 2024-04-25T01:09:02.862 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T01:09:02.862 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:09:02.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:09:02.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:02.922 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T01:09:02.922 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:09:02.922 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:02.970 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T01:09:02.970 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:09:02.971 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:03.019 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:09:03.076 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:09:03.124 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:09:03.125 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:09:03.126 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:09:03.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:03.127 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:09:03.178 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T01:09:03.179 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T01:09:03.179 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T01:09:03.223 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:09:03.237 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:03.282 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:09:03.282 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:09:03.282 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:09:03.282 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:09:03.282 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T01:09:03.283 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:03.284 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:09:03.285 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:09:03.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:03.285 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:09:03.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:2024-04-25T01:09:03.308+0000 7f249adec580 -1 init, newargv = 0x5652ce6e9aa0 newargc=15 2024-04-25T01:09:03.304 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:ceph-fuse[33302]: starting ceph client 2024-04-25T01:09:03.308 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T01:09:03.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi138.stderr:ceph-fuse[33302]: starting fuse 2024-04-25T01:09:04.411 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:09:04.430 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:04.484 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:09:04.484 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:09:04.484 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:09:04.484 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:09:04.484 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/run/lock 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:09:04.485 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/fs/nfsd 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/run/rpc_pipefs 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T01:09:04.486 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T01:09:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:09:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:09:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:09:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:09:04.487 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:04.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:04.488 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:09:04.535 INFO:teuthology.orchestra.run.smithi138.stdout:45 2024-04-25T01:09:04.535 INFO:teuthology.orchestra.run.smithi138.stdout:46 2024-04-25T01:09:04.535 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T01:09:04.535 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:09:04.535 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:04.536 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.3")) 2024-04-25T01:09:04.537 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-04-25T01:09:04.660 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.3.33302.asok 2024-04-25T01:09:04.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:09:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:04.674 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33302.asok 2024-04-25T01:09:04.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:04.674 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33302.asok status 2024-04-25T01:09:04.826 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T01:09:04.826 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-04-25T01:09:04.826 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:09:04.826 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "3", 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "33302", 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4550, 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 4550 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T01:09:04.827 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1499145833 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.2:0", 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1499145833 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.4550 192.168.0.2:0/1499145833", 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.2:0/1499145833", 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 9, 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 25, 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-04-25T01:09:04.828 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-04-25T01:09:04.829 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T01:09:04.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:04.843 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-25T01:09:04.843 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:04.851 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-25T01:09:04.851 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:04.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:04.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:04.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:04.906 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-25T01:09:04.907 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T01:09:04.953 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-25T01:09:04.953 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T01:09:04.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:04.954 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T01:09:05.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:05.013 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-04-25T01:09:05.013 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:05.020 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-04-25T01:09:05.020 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:05.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:05.020 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:05.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:05.075 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-04-25T01:09:05.075 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:05.123 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-04-25T01:09:05.124 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:05.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:05.124 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:05.178 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T01:09:05.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.337+0000 7f5642537640 1 Processor -- start 2024-04-25T01:09:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.337+0000 7f5642537640 1 -- start start 2024-04-25T01:09:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5642537640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c15e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5642537640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f563c15ea20 0x7f563c0b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5642537640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563c147a20 con 0x7f563c147640 2024-04-25T01:09:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5642537640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f563c1627e0 con 0x7f563c15ea20 2024-04-25T01:09:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5642537640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f563c15f230 con 0x7f563c06e1c0 2024-04-25T01:09:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f563c15ea20 0x7f563c0b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f563c15ea20 0x7f563c0b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51304/0 (socket says 172.21.15.33:51304) 2024-04-25T01:09:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 -- 172.21.15.33:0/3688229733 learned_addr learned my addr 172.21.15.33:0/3688229733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f563b7fe640 1 --2- 172.21.15.33:0/3688229733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c15e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 -- 172.21.15.33:0/3688229733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 msgr2=0x7f563c15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 --2- 172.21.15.33:0/3688229733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c15e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 -- 172.21.15.33:0/3688229733 >> v1:172.21.15.138:6789/0 conn(0x7f563c147640 legacy=0x7f563c06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 -- 172.21.15.33:0/3688229733 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c161a20 con 0x7f563c15ea20 2024-04-25T01:09:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f5640ad4640 1 --2- 172.21.15.33:0/3688229733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c0b4840 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5634009e50 tx=0x7f563401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5963426221aafa43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.341+0000 7f563affd640 1 -- 172.21.15.33:0/3688229733 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340a5830 con 0x7f563c15ea20 2024-04-25T01:09:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f563affd640 1 -- 172.21.15.33:0/3688229733 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56340a59d0 con 0x7f563c15ea20 2024-04-25T01:09:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f563affd640 1 -- 172.21.15.33:0/3688229733 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340a5da0 con 0x7f563c15ea20 2024-04-25T01:09:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- 172.21.15.33:0/3688229733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 msgr2=0x7f563c0b4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 --2- 172.21.15.33:0/3688229733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c0b4840 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5634009e50 tx=0x7f563401c970 comp rx=0 tx=0).stop 2024-04-25T01:09:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- 172.21.15.33:0/3688229733 shutdown_connections 2024-04-25T01:09:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 --2- 172.21.15.33:0/3688229733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c0b4840 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 --2- 172.21.15.33:0/3688229733 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c15e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- 172.21.15.33:0/3688229733 >> 172.21.15.33:0/3688229733 conn(0x7f563c0b1810 msgr2=0x7f563c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:05.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- 172.21.15.33:0/3688229733 shutdown_connections 2024-04-25T01:09:05.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- 172.21.15.33:0/3688229733 wait complete. 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 Processor -- start 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- start start 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c0af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563c147640 0x7f563c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c15ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f563c0b5590 con 0x7f563c15ea20 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f563c15eed0 con 0x7f563c147640 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f563c0b5710 con 0x7f563c06e1c0 2024-04-25T01:09:05.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c15ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c15ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51308/0 (socket says 172.21.15.33:51308) 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 -- 172.21.15.33:0/952471533 learned_addr learned my addr 172.21.15.33:0/952471533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f563b7fe640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563c147640 0x7f563c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f563bfff640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c0af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 -- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 msgr2=0x7f563c0af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c0af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 -- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563c147640 msgr2=0x7f563c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563c147640 0x7f563c15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 -- 172.21.15.33:0/952471533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c15ff70 con 0x7f563c15ea20 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f563b7fe640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563c147640 0x7f563c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f563bfff640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c0af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5640ad4640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c15ce30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f56340af040 tx=0x7f5634019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5638ff9640 1 -- 172.21.15.33:0/952471533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340bc020 con 0x7f563c15ea20 2024-04-25T01:09:05.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5638ff9640 1 -- 172.21.15.33:0/952471533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56340b8670 con 0x7f563c15ea20 2024-04-25T01:09:05.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5638ff9640 1 -- 172.21.15.33:0/952471533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56340b8a40 con 0x7f563c15ea20 2024-04-25T01:09:05.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c160250 con 0x7f563c15ea20 2024-04-25T01:09:05.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.345+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c160530 con 0x7f563c15ea20 2024-04-25T01:09:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.349+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c069040 con 0x7f563c15ea20 2024-04-25T01:09:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.349+0000 7f5638ff9640 1 -- 172.21.15.33:0/952471533 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f56340bf020 con 0x7f563c15ea20 2024-04-25T01:09:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.349+0000 7f5638ff9640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56280678f0 0x7f5628069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.349+0000 7f5638ff9640 1 -- 172.21.15.33:0/952471533 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5634120fa0 con 0x7f563c15ea20 2024-04-25T01:09:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.349+0000 7f563bfff640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56280678f0 0x7f5628069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.349+0000 7f563bfff640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56280678f0 0x7f5628069db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f563c147060 tx=0x7f5630009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.353+0000 7f5638ff9640 1 -- 172.21.15.33:0/952471533 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f56340c6030 con 0x7f563c15ea20 2024-04-25T01:09:05.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.517+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f563c159930 con 0x7f563c15ea20 2024-04-25T01:09:05.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.521+0000 7f5638ff9640 1 -- 172.21.15.33:0/952471533 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f5634016620 con 0x7f563c15ea20 2024-04-25T01:09:05.513 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:05.513 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56280678f0 msgr2=0x7f5628069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56280678f0 0x7f5628069db0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f563c147060 tx=0x7f5630009040 comp rx=0 tx=0).stop 2024-04-25T01:09:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 msgr2=0x7f563c15ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c15ce30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f56340af040 tx=0x7f5634019710 comp rx=0 tx=0).stop 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 shutdown_connections 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f56280678f0 0x7f5628069db0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f563c15ea20 0x7f563c15ce30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f563c147640 0x7f563c15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 --2- 172.21.15.33:0/952471533 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f563c06e1c0 0x7f563c0af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 >> 172.21.15.33:0/952471533 conn(0x7f563c0b1810 msgr2=0x7f563c0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 shutdown_connections 2024-04-25T01:09:05.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.525+0000 7f5642537640 1 -- 172.21.15.33:0/952471533 wait complete. 2024-04-25T01:09:05.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:05.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.665+0000 7f11a19ab640 1 Processor -- start 2024-04-25T01:09:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f11a19ab640 1 -- start start 2024-04-25T01:09:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f11a19ab640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f119c0b5580 0x7f119c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f11a19ab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0b50d0 0x7f119c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f11a19ab640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f119c0b05a0 con 0x7f119c0d1b50 2024-04-25T01:09:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f11a19ab640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f119c0b5ea0 con 0x7f119c0b5580 2024-04-25T01:09:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f11a19ab640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f119c15d380 con 0x7f119c0b50d0 2024-04-25T01:09:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f119c0b5580 0x7f119c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f119c0b5580 0x7f119c0b5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51314/0 (socket says 172.21.15.33:51314) 2024-04-25T01:09:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119affd640 1 -- 172.21.15.33:0/4038587244 learned_addr learned my addr 172.21.15.33:0/4038587244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119a7fc640 1 --2- 172.21.15.33:0/4038587244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0b50d0 0x7f119c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119affd640 1 -- 172.21.15.33:0/4038587244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0b50d0 msgr2=0x7f119c15ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119affd640 1 --2- 172.21.15.33:0/4038587244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0b50d0 0x7f119c15ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119affd640 1 -- 172.21.15.33:0/4038587244 >> v1:172.21.15.138:6789/0 conn(0x7f119c0d1b50 legacy=0x7f119c0d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.669+0000 7f119affd640 1 -- 172.21.15.33:0/4038587244 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c0af500 con 0x7f119c0b5580 2024-04-25T01:09:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 --2- 172.21.15.33:0/4038587244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b5580 0x7f119c0b5960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f118c009e50 tx=0x7f118c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e3b2ea90620810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f1199ffb640 1 -- 172.21.15.33:0/4038587244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0a56e0 con 0x7f119c0b5580 2024-04-25T01:09:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f1199ffb640 1 -- 172.21.15.33:0/4038587244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c0a5880 con 0x7f119c0b5580 2024-04-25T01:09:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f1199ffb640 1 -- 172.21.15.33:0/4038587244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0a5c50 con 0x7f119c0b5580 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- 172.21.15.33:0/4038587244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b5580 msgr2=0x7f119c0b5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 --2- 172.21.15.33:0/4038587244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b5580 0x7f119c0b5960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f118c009e50 tx=0x7f118c01ca30 comp rx=0 tx=0).stop 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- 172.21.15.33:0/4038587244 shutdown_connections 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 --2- 172.21.15.33:0/4038587244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0b50d0 0x7f119c15ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 --2- 172.21.15.33:0/4038587244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b5580 0x7f119c0b5960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- 172.21.15.33:0/4038587244 >> 172.21.15.33:0/4038587244 conn(0x7f119c09a9d0 msgr2=0x7f119c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- 172.21.15.33:0/4038587244 shutdown_connections 2024-04-25T01:09:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- 172.21.15.33:0/4038587244 wait complete. 2024-04-25T01:09:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 Processor -- start 2024-04-25T01:09:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- start start 2024-04-25T01:09:05.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b50d0 0x7f119c15b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f119c0b5580 0x7f119c146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0d1b50 0x7f119c148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f119c0af1b0 con 0x7f119c0b50d0 2024-04-25T01:09:05.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f119c0b0080 con 0x7f119c0b5580 2024-04-25T01:09:05.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f11a19ab640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f119c15e5e0 con 0x7f119c0d1b50 2024-04-25T01:09:05.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b50d0 0x7f119c15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b50d0 0x7f119c15b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51330/0 (socket says 172.21.15.33:51330) 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 -- 172.21.15.33:0/2966300784 learned_addr learned my addr 172.21.15.33:0/2966300784 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119a7fc640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f119c0b5580 0x7f119c146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119b7fe640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0d1b50 0x7f119c148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 -- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0d1b50 msgr2=0x7f119c148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0d1b50 0x7f119c148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 -- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f119c0b5580 msgr2=0x7f119c146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f119c0b5580 0x7f119c146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.673+0000 7f119affd640 1 -- 172.21.15.33:0/2966300784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f119c146a40 con 0x7f119c0b50d0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f119affd640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b50d0 0x7f119c15b130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f118c0af040 tx=0x7f118c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f1193fff640 1 -- 172.21.15.33:0/2966300784 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0b8670 con 0x7f119c0b50d0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f1193fff640 1 -- 172.21.15.33:0/2966300784 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f118c0be070 con 0x7f119c0b50d0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f119c147840 con 0x7f119c0b50d0 2024-04-25T01:09:05.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f1193fff640 1 -- 172.21.15.33:0/2966300784 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118c0b8a70 con 0x7f119c0b50d0 2024-04-25T01:09:05.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f119c146fa0 con 0x7f119c0b50d0 2024-04-25T01:09:05.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119c069040 con 0x7f119c0b50d0 2024-04-25T01:09:05.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f1193fff640 1 -- 172.21.15.33:0/2966300784 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f118c0b8c10 con 0x7f119c0b50d0 2024-04-25T01:09:05.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f1193fff640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f117c0678c0 0x7f117c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f1193fff640 1 -- 172.21.15.33:0/2966300784 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f118c121b10 con 0x7f119c0b50d0 2024-04-25T01:09:05.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.677+0000 7f119a7fc640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f117c0678c0 0x7f117c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.681+0000 7f119a7fc640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f117c0678c0 0x7f117c069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f119c14e060 tx=0x7f1184008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.681+0000 7f1193fff640 1 -- 172.21.15.33:0/2966300784 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f119c146a40 con 0x7f119c0b50d0 2024-04-25T01:09:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.841+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f119c0d1930 con 0x7f119c0b50d0 2024-04-25T01:09:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f1193fff640 1 -- 172.21.15.33:0/2966300784 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f118c016620 con 0x7f119c0b50d0 2024-04-25T01:09:05.836 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f117c0678c0 msgr2=0x7f117c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f117c0678c0 0x7f117c069d80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f119c14e060 tx=0x7f1184008040 comp rx=0 tx=0).stop 2024-04-25T01:09:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b50d0 msgr2=0x7f119c15b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b50d0 0x7f119c15b130 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f118c0af040 tx=0x7f118c0197d0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 shutdown_connections 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f117c0678c0 0x7f117c069d80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f119c0d1b50 0x7f119c148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f119c0b5580 0x7f119c146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 --2- 172.21.15.33:0/2966300784 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f119c0b50d0 0x7f119c15b130 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 >> 172.21.15.33:0/2966300784 conn(0x7f119c09a9d0 msgr2=0x7f119c150400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 shutdown_connections 2024-04-25T01:09:05.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.845+0000 7f11a19ab640 1 -- 172.21.15.33:0/2966300784 wait complete. 2024-04-25T01:09:05.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:05.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 Processor -- start 2024-04-25T01:09:05.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 -- start start 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e6c0b5630 0x7f7e6c161ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e6c0b5a90 0x7f7e6c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f7e6c06e330 con 0x7f7e6c0b5630 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7e6c0b5f60 con 0x7f7e6c0b5a90 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e73edb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c0b6240 con 0x7f7e6c158480 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e6c0b5630 0x7f7e6c161ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7e6c0b5630 0x7f7e6c161ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51334/0 (socket says 172.21.15.33:51334) 2024-04-25T01:09:05.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 -- 172.21.15.33:0/3467001244 learned_addr learned my addr 172.21.15.33:0/3467001244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71c77640 1 --2- 172.21.15.33:0/3467001244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e72478640 1 --2- 172.21.15.33:0/3467001244 >> v2:172.21.15.138:3300/0 conn(0x7f7e6c0b5a90 0x7f7e6c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 -- 172.21.15.33:0/3467001244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 msgr2=0x7f7e6c070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 --2- 172.21.15.33:0/3467001244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 -- 172.21.15.33:0/3467001244 >> v2:172.21.15.138:3300/0 conn(0x7f7e6c0b5a90 msgr2=0x7f7e6c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 --2- 172.21.15.33:0/3467001244 >> v2:172.21.15.138:3300/0 conn(0x7f7e6c0b5a90 0x7f7e6c161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.993+0000 7f7e71476640 1 -- 172.21.15.33:0/3467001244 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c0b6520 con 0x7f7e6c0b5630 2024-04-25T01:09:05.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e71476640 1 --2- 172.21.15.33:0/3467001244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c161ab0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0d2890 tx=0x7f7e5c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a48c685d6cdb52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e70c75640 1 -- 172.21.15.33:0/3467001244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0aa610 con 0x7f7e6c0b5630 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e70c75640 1 -- 172.21.15.33:0/3467001244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0af070 con 0x7f7e6c0b5630 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e70c75640 1 -- 172.21.15.33:0/3467001244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0aa9d0 con 0x7f7e6c0b5630 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 -- 172.21.15.33:0/3467001244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 msgr2=0x7f7e6c161ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 --2- 172.21.15.33:0/3467001244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c161ab0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0d2890 tx=0x7f7e5c01c4f0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 -- 172.21.15.33:0/3467001244 shutdown_connections 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 --2- 172.21.15.33:0/3467001244 >> v2:172.21.15.138:3300/0 conn(0x7f7e6c0b5a90 0x7f7e6c161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 --2- 172.21.15.33:0/3467001244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c161ab0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 --2- 172.21.15.33:0/3467001244 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 -- 172.21.15.33:0/3467001244 >> 172.21.15.33:0/3467001244 conn(0x7f7e6c0734c0 msgr2=0x7f7e6c09a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 -- 172.21.15.33:0/3467001244 shutdown_connections 2024-04-25T01:09:05.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 -- 172.21.15.33:0/3467001244 wait complete. 2024-04-25T01:09:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 Processor -- start 2024-04-25T01:09:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:05.997+0000 7f7e73edb640 1 -- start start 2024-04-25T01:09:05.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e6c0b5a90 0x7f7e6c0afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c15ee20 con 0x7f7e6c0b5630 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c15eca0 con 0x7f7e6c0b5a90 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c15efa0 con 0x7f7e6c158480 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c155ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51346/0 (socket says 172.21.15.33:51346) 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 -- 172.21.15.33:0/1003192247 learned_addr learned my addr 172.21.15.33:0/1003192247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e72478640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71476640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e6c0b5a90 0x7f7e6c0afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 -- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 msgr2=0x7f7e6c150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 -- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e6c0b5a90 msgr2=0x7f7e6c0afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:05.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e6c0b5a90 0x7f7e6c0afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 -- 172.21.15.33:0/1003192247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c1535a0 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71476640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e6c0b5a90 0x7f7e6c0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71c77640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c155ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d5f0 tx=0x7f7e6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e627fc640 1 -- 172.21.15.33:0/1003192247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680125f0 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e627fc640 1 -- 172.21.15.33:0/1003192247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e68019070 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c153880 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e627fc640 1 -- 172.21.15.33:0/1003192247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e680129b0 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c09e3f0 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c069040 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e627fc640 1 -- 172.21.15.33:0/1003192247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7e68012b50 con 0x7f7e6c0b5630 2024-04-25T01:09:05.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e627fc640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7e500678c0 0x7f7e50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:05.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e627fc640 1 -- 172.21.15.33:0/1003192247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e68086360 con 0x7f7e6c0b5630 2024-04-25T01:09:05.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.001+0000 7f7e71476640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7e500678c0 0x7f7e50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:05.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.005+0000 7f7e71476640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7e500678c0 0x7f7e50069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ae000 tx=0x7f7e5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:05.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.005+0000 7f7e627fc640 1 -- 172.21.15.33:0/1003192247 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7e68056bf0 con 0x7f7e6c0b5630 2024-04-25T01:09:06.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.165+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7e6c0b7050 con 0x7f7e6c0b5630 2024-04-25T01:09:06.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.165+0000 7f7e627fc640 1 -- 172.21.15.33:0/1003192247 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f7e680564f0 con 0x7f7e6c0b5630 2024-04-25T01:09:06.159 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7e500678c0 msgr2=0x7f7e50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7e500678c0 0x7f7e50069d80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0ae000 tx=0x7f7e5c008040 comp rx=0 tx=0).stop 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 msgr2=0x7f7e6c155ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c155ee0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800d5f0 tx=0x7f7e6800dab0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 shutdown_connections 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7e500678c0 0x7f7e50069d80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7e6c158480 0x7f7e6c150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e6c0b5a90 0x7f7e6c0afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 --2- 172.21.15.33:0/1003192247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e6c0b5630 0x7f7e6c155ee0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 >> 172.21.15.33:0/1003192247 conn(0x7f7e6c0734c0 msgr2=0x7f7e6c0b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:06.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 shutdown_connections 2024-04-25T01:09:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.169+0000 7f7e73edb640 1 -- 172.21.15.33:0/1003192247 wait complete. 2024-04-25T01:09:06.176 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd491b09640 1 Processor -- start 2024-04-25T01:09:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd491b09640 1 -- start start 2024-04-25T01:09:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd491b09640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd48c156600 0x7fd48c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd491b09640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c15cfb0 0x7fd48c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd491b09640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd48c070860 con 0x7fd48c0d2140 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd491b09640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fd48c06f440 con 0x7fd48c156600 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd491b09640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c0d2590 con 0x7fd48c15cfb0 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48b7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd48c156600 0x7fd48c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48bfff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd48c0d2140 0x7fd48c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39114/0 (socket says 172.21.15.33:39114) 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48bfff640 1 -- 172.21.15.33:0/2532742488 learned_addr learned my addr 172.21.15.33:0/2532742488 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48affd640 1 --2- 172.21.15.33:0/2532742488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c15cfb0 0x7fd48c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48b7fe640 1 -- 172.21.15.33:0/2532742488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c15cfb0 msgr2=0x7fd48c0d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/2532742488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c15cfb0 0x7fd48c0d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48b7fe640 1 -- 172.21.15.33:0/2532742488 >> v1:172.21.15.33:6789/0 conn(0x7fd48c0d2140 legacy=0x7fd48c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.321+0000 7fd48b7fe640 1 -- 172.21.15.33:0/2532742488 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c1569e0 con 0x7fd48c156600 2024-04-25T01:09:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/2532742488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c156600 0x7fd48c06ef00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097c0 tx=0x7fd47401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac54be34add696a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd48a7fc640 1 -- 172.21.15.33:0/2532742488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740b0070 con 0x7fd48c156600 2024-04-25T01:09:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd48a7fc640 1 -- 172.21.15.33:0/2532742488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4740ab600 con 0x7fd48c156600 2024-04-25T01:09:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd48a7fc640 1 -- 172.21.15.33:0/2532742488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4740ab9d0 con 0x7fd48c156600 2024-04-25T01:09:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- 172.21.15.33:0/2532742488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c156600 msgr2=0x7fd48c06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 --2- 172.21.15.33:0/2532742488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c156600 0x7fd48c06ef00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd4740097c0 tx=0x7fd47401c410 comp rx=0 tx=0).stop 2024-04-25T01:09:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- 172.21.15.33:0/2532742488 shutdown_connections 2024-04-25T01:09:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 --2- 172.21.15.33:0/2532742488 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c15cfb0 0x7fd48c0d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 --2- 172.21.15.33:0/2532742488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c156600 0x7fd48c06ef00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- 172.21.15.33:0/2532742488 >> 172.21.15.33:0/2532742488 conn(0x7fd48c09a9d0 msgr2=0x7fd48c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- 172.21.15.33:0/2532742488 shutdown_connections 2024-04-25T01:09:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- 172.21.15.33:0/2532742488 wait complete. 2024-04-25T01:09:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 Processor -- start 2024-04-25T01:09:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- start start 2024-04-25T01:09:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c0d2140 0x7fd48c0b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c156600 0x7fd48c0b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c15cfb0 0x7fd48c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0d2a60 con 0x7fd48c15cfb0 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c0d28e0 con 0x7fd48c0d2140 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd491b09640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd48c0d0a90 con 0x7fd48c156600 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.325+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c15cfb0 0x7fd48c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c15cfb0 0x7fd48c1555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51376/0 (socket says 172.21.15.33:51376) 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 -- 172.21.15.33:0/3905484102 learned_addr learned my addr 172.21.15.33:0/3905484102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48affd640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c156600 0x7fd48c0b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c0d2140 0x7fd48c0b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 -- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c156600 msgr2=0x7fd48c0b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c156600 0x7fd48c0b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 -- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c0d2140 msgr2=0x7fd48c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c0d2140 0x7fd48c0b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 -- 172.21.15.33:0/3905484102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c0b36f0 con 0x7fd48c15cfb0 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48bfff640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c15cfb0 0x7fd48c1555a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c390 tx=0x7fd48400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd483fff640 1 -- 172.21.15.33:0/3905484102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484017070 con 0x7fd48c15cfb0 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd483fff640 1 -- 172.21.15.33:0/3905484102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd484005cf0 con 0x7fd48c15cfb0 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd483fff640 1 -- 172.21.15.33:0/3905484102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4840072f0 con 0x7fd48c15cfb0 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c1547e0 con 0x7fd48c15cfb0 2024-04-25T01:09:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c154da0 con 0x7fd48c15cfb0 2024-04-25T01:09:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c069040 con 0x7fd48c15cfb0 2024-04-25T01:09:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd483fff640 1 -- 172.21.15.33:0/3905484102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd48401a070 con 0x7fd48c15cfb0 2024-04-25T01:09:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd483fff640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd46c0678c0 0x7fd46c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd483fff640 1 -- 172.21.15.33:0/3905484102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd4840852a0 con 0x7fd48c15cfb0 2024-04-25T01:09:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.329+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd46c0678c0 0x7fd46c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.333+0000 7fd48b7fe640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd46c0678c0 0x7fd46c069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ad000 tx=0x7fd474008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.333+0000 7fd483fff640 1 -- 172.21.15.33:0/3905484102 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd484055b30 con 0x7fd48c15cfb0 2024-04-25T01:09:06.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.497+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd48c0aef40 con 0x7fd48c15cfb0 2024-04-25T01:09:06.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.497+0000 7fd483fff640 1 -- 172.21.15.33:0/3905484102 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fd484055430 con 0x7fd48c15cfb0 2024-04-25T01:09:06.491 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd46c0678c0 msgr2=0x7fd46c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd46c0678c0 0x7fd46c069d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4740ad000 tx=0x7fd474008040 comp rx=0 tx=0).stop 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c15cfb0 msgr2=0x7fd48c1555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c15cfb0 0x7fd48c1555a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd48400c390 tx=0x7fd48400c850 comp rx=0 tx=0).stop 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 shutdown_connections 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd46c0678c0 0x7fd46c069d80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c15cfb0 0x7fd48c1555a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd48c156600 0x7fd48c0b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 --2- 172.21.15.33:0/3905484102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c0d2140 0x7fd48c0b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 >> 172.21.15.33:0/3905484102 conn(0x7fd48c09a9d0 msgr2=0x7fd48c0af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 shutdown_connections 2024-04-25T01:09:06.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.501+0000 7fd491b09640 1 -- 172.21.15.33:0/3905484102 wait complete. 2024-04-25T01:09:06.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8b8b0640 1 Processor -- start 2024-04-25T01:09:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8b8b0640 1 -- start start 2024-04-25T01:09:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8b8b0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 0x7ffa840d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8b8b0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa840d4b30 con 0x7ffa8415e7d0 2024-04-25T01:09:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8b8b0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa840cf7e0 con 0x7ffa840b4ae0 2024-04-25T01:09:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8b8b0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840d48e0 con 0x7ffa8415e3f0 2024-04-25T01:09:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa88e4b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffa8415e7d0 0x7ffa840d24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39118/0 (socket says 172.21.15.33:39118) 2024-04-25T01:09:06.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa88e4b640 1 -- 172.21.15.33:0/2254228051 learned_addr learned my addr 172.21.15.33:0/2254228051 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8964c640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 0x7ffa840d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2251170834 0 0) 0x7ffa840d4b30 con 0x7ffa8415e7d0 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa70003530 con 0x7ffa8415e7d0 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2772643503 0 0) 0x7ffa70003530 con 0x7ffa8415e7d0 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 msgr2=0x7ffa840d1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 0x7ffa840d1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 >> v1:172.21.15.138:6789/0 conn(0x7ffa840b4ae0 legacy=0x7ffa840c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa840cfac0 con 0x7ffa8415e7d0 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7ffa74001680 con 0x7ffa8415e7d0 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 >> v1:172.21.15.33:6789/0 conn(0x7ffa8415e7d0 legacy=0x7ffa840d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa70004810 0x7ffa70004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 0x7ffa70005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840d48e0 con 0x7ffa700039f0 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7000bb60 con 0x7ffa70004810 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa7000bd60 con 0x7ffa8415e3f0 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8964c640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa70003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa88e4b640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa70004810 0x7ffa70004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8964c640 1 -- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 msgr2=0x7ffa70005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa89e4d640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 0x7ffa70005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8964c640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 0x7ffa70005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8964c640 1 -- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa70004810 msgr2=0x7ffa70004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8964c640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa70004810 0x7ffa70004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa8964c640 1 -- 172.21.15.33:0/2254228051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa70003530 con 0x7ffa700039f0 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa88e4b640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa70004810 0x7ffa70004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:06.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.645+0000 7ffa89e4d640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa8415e3f0 0x7ffa70005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8964c640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa70003e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009ac0 tx=0x7ffa8001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2374302e343e1ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa800b2070 con 0x7ffa700039f0 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa800a5530 con 0x7ffa700039f0 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa7bfff640 1 -- 172.21.15.33:0/2254228051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80009690 con 0x7ffa700039f0 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/2254228051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 msgr2=0x7ffa70003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 --2- 172.21.15.33:0/2254228051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa70003e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009ac0 tx=0x7ffa8001ca50 comp rx=0 tx=0).stop 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa89e4d640 1 -- 172.21.15.33:0/2254228051 reap_dead start 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/2254228051 shutdown_connections 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/2254228051 >> 172.21.15.33:0/2254228051 conn(0x7ffa840d3dc0 msgr2=0x7ffa840d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/2254228051 shutdown_connections 2024-04-25T01:09:06.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/2254228051 wait complete. 2024-04-25T01:09:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 Processor -- start 2024-04-25T01:09:06.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.649+0000 7ffa8b8b0640 1 -- start start 2024-04-25T01:09:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa841f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa84160b40 0x7ffa8414bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa84160fc0 0x7ffa8414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840c8bc0 con 0x7ffa700039f0 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa840c8a40 con 0x7ffa84160fc0 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffa840d2dc0 con 0x7ffa84160b40 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa841f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa841f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51408/0 (socket says 172.21.15.33:51408) 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 -- 172.21.15.33:0/3194810444 learned_addr learned my addr 172.21.15.33:0/3194810444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa88e4b640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa84160b40 0x7ffa8414bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa89e4d640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa84160fc0 0x7ffa8414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 -- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa84160b40 msgr2=0x7ffa8414bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa84160b40 0x7ffa8414bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 -- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa84160fc0 msgr2=0x7ffa8414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa84160fc0 0x7ffa8414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 -- 172.21.15.33:0/3194810444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa84160110 con 0x7ffa700039f0 2024-04-25T01:09:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8964c640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa841f7a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa800b0040 tx=0x7ffa70005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa79ffb640 1 -- 172.21.15.33:0/3194810444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa800b2040 con 0x7ffa700039f0 2024-04-25T01:09:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa79ffb640 1 -- 172.21.15.33:0/3194810444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8000f000 con 0x7ffa700039f0 2024-04-25T01:09:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8415f070 con 0x7ffa700039f0 2024-04-25T01:09:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa79ffb640 1 -- 172.21.15.33:0/3194810444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa800b7680 con 0x7ffa700039f0 2024-04-25T01:09:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8415f630 con 0x7ffa700039f0 2024-04-25T01:09:06.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa84069040 con 0x7ffa700039f0 2024-04-25T01:09:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa79ffb640 1 -- 172.21.15.33:0/3194810444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffa800c0030 con 0x7ffa700039f0 2024-04-25T01:09:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa79ffb640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffa680678c0 0x7ffa68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa79ffb640 1 -- 172.21.15.33:0/3194810444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffa8011f830 con 0x7ffa700039f0 2024-04-25T01:09:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.653+0000 7ffa88e4b640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffa680678c0 0x7ffa68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.657+0000 7ffa88e4b640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffa680678c0 0x7ffa68069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffa84159410 tx=0x7ffa74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.657+0000 7ffa79ffb640 1 -- 172.21.15.33:0/3194810444 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffa80016760 con 0x7ffa700039f0 2024-04-25T01:09:06.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffa8408d5a0 con 0x7ffa700039f0 2024-04-25T01:09:06.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa79ffb640 1 -- 172.21.15.33:0/3194810444 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7ffa800efa70 con 0x7ffa700039f0 2024-04-25T01:09:06.813 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffa680678c0 msgr2=0x7ffa68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffa680678c0 0x7ffa68069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffa84159410 tx=0x7ffa74009040 comp rx=0 tx=0).stop 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 msgr2=0x7ffa841f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa841f7a50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa800b0040 tx=0x7ffa70005840 comp rx=0 tx=0).stop 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 shutdown_connections 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffa680678c0 0x7ffa68069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa84160fc0 0x7ffa8414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffa84160b40 0x7ffa8414bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 --2- 172.21.15.33:0/3194810444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa700039f0 0x7ffa841f7a50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 >> 172.21.15.33:0/3194810444 conn(0x7ffa840d3dc0 msgr2=0x7ffa840d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:06.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 shutdown_connections 2024-04-25T01:09:06.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.821+0000 7ffa8b8b0640 1 -- 172.21.15.33:0/3194810444 wait complete. 2024-04-25T01:09:06.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:06.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e461e4640 1 Processor -- start 2024-04-25T01:09:06.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e461e4640 1 -- start start 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e461e4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e401444f0 0x7f0e400b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e461e4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e40072540 0x7f0e400ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e461e4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e401448d0 con 0x7f0e4015e660 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e461e4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0e400ce0c0 con 0x7f0e401444f0 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e461e4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0e400b5b60 con 0x7f0e40072540 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3f7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e401444f0 0x7f0e400b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3f7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0e401444f0 0x7f0e400b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51414/0 (socket says 172.21.15.33:51414) 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/2971635961 learned_addr learned my addr 172.21.15.33:0/2971635961 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:06.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3ffff640 1 --2- 172.21.15.33:0/2971635961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e40072540 0x7f0e400ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/2971635961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e40072540 msgr2=0x7f0e400ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:06.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3f7fe640 1 --2- 172.21.15.33:0/2971635961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e40072540 0x7f0e400ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/2971635961 >> v1:172.21.15.138:6789/0 conn(0x7f0e4015e660 legacy=0x7f0e40071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:06.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.965+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/2971635961 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e4015ec20 con 0x7f0e401444f0 2024-04-25T01:09:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e3f7fe640 1 --2- 172.21.15.33:0/2971635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401444f0 0x7f0e400b5620 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009e50 tx=0x7f0e3001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6fe76b31c85927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e3e7fc640 1 -- 172.21.15.33:0/2971635961 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300ac020 con 0x7f0e401444f0 2024-04-25T01:09:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e3e7fc640 1 -- 172.21.15.33:0/2971635961 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e300a6410 con 0x7f0e401444f0 2024-04-25T01:09:06.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e3e7fc640 1 -- 172.21.15.33:0/2971635961 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300a67e0 con 0x7f0e401444f0 2024-04-25T01:09:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 -- 172.21.15.33:0/2971635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401444f0 msgr2=0x7f0e400b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 --2- 172.21.15.33:0/2971635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401444f0 0x7f0e400b5620 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0e30009e50 tx=0x7f0e3001c780 comp rx=0 tx=0).stop 2024-04-25T01:09:06.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 -- 172.21.15.33:0/2971635961 shutdown_connections 2024-04-25T01:09:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 --2- 172.21.15.33:0/2971635961 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e40072540 0x7f0e400ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 --2- 172.21.15.33:0/2971635961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e401444f0 0x7f0e400b5620 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 -- 172.21.15.33:0/2971635961 >> 172.21.15.33:0/2971635961 conn(0x7f0e400b1810 msgr2=0x7f0e400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 -- 172.21.15.33:0/2971635961 shutdown_connections 2024-04-25T01:09:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 -- 172.21.15.33:0/2971635961 wait complete. 2024-04-25T01:09:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 Processor -- start 2024-04-25T01:09:06.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.969+0000 7f0e461e4640 1 -- start start 2024-04-25T01:09:06.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e40072540 0x7f0e40154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e401444f0 0x7f0e40138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e4015e660 0x7f0e400b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400b37a0 con 0x7f0e40072540 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0e400b3620 con 0x7f0e4015e660 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0e40167a00 con 0x7f0e401444f0 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e40072540 0x7f0e40154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e40072540 0x7f0e40154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51430/0 (socket says 172.21.15.33:51430) 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/1482812405 learned_addr learned my addr 172.21.15.33:0/1482812405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3effd640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e401444f0 0x7f0e40138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3ffff640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e4015e660 0x7f0e400b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e401444f0 msgr2=0x7f0e40138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e401444f0 0x7f0e40138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e4015e660 msgr2=0x7f0e400b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e4015e660 0x7f0e400b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 -- 172.21.15.33:0/1482812405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e40141d30 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3f7fe640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e40072540 0x7f0e40154e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0e3001d020 tx=0x7f0e30019520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e27fff640 1 -- 172.21.15.33:0/1482812405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e300ac040 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e27fff640 1 -- 172.21.15.33:0/1482812405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e300b1430 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e40141fd0 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e27fff640 1 -- 172.21.15.33:0/1482812405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e30009510 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e40142590 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e40069040 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e27fff640 1 -- 172.21.15.33:0/1482812405 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0e300af020 con 0x7f0e40072540 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e27fff640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0e2c0678c0 0x7f0e2c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:06.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e27fff640 1 -- 172.21.15.33:0/1482812405 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0e301205b0 con 0x7f0e40072540 2024-04-25T01:09:06.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.973+0000 7f0e3effd640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0e2c0678c0 0x7f0e2c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:06.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.977+0000 7f0e3effd640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0e2c0678c0 0x7f0e2c069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e400b14a0 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:06.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:06.977+0000 7f0e27fff640 1 -- 172.21.15.33:0/1482812405 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0e300c1060 con 0x7f0e40072540 2024-04-25T01:09:07.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.145+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e400b2f20 con 0x7f0e40072540 2024-04-25T01:09:07.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.145+0000 7f0e27fff640 1 -- 172.21.15.33:0/1482812405 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f0e401448d0 con 0x7f0e40072540 2024-04-25T01:09:07.138 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:07.138 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0e2c0678c0 msgr2=0x7f0e2c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0e2c0678c0 0x7f0e2c069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e400b14a0 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2024-04-25T01:09:07.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e40072540 msgr2=0x7f0e40154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e40072540 0x7f0e40154e10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0e3001d020 tx=0x7f0e30019520 comp rx=0 tx=0).stop 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 shutdown_connections 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0e2c0678c0 0x7f0e2c069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e4015e660 0x7f0e400b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0e401444f0 0x7f0e40138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 --2- 172.21.15.33:0/1482812405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e40072540 0x7f0e40154e10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 >> 172.21.15.33:0/1482812405 conn(0x7f0e400b1810 msgr2=0x7f0e4015a050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 shutdown_connections 2024-04-25T01:09:07.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.149+0000 7f0e461e4640 1 -- 172.21.15.33:0/1482812405 wait complete. 2024-04-25T01:09:07.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:07.158 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028c9a2640 1 Processor -- start 2024-04-25T01:09:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028c9a2640 1 -- start start 2024-04-25T01:09:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028c9a2640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f028815e0d0 0x7f02880cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028c9a2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d2ac0 0x7f0288167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028c9a2640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02880b6cc0 con 0x7f02880d1f60 2024-04-25T01:09:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028c9a2640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f02880b7580 con 0x7f028815e0d0 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028c9a2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f02880b6f30 con 0x7f02880d2ac0 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028659c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f028815e0d0 0x7f02880cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f0285d9b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f02880d1f60 0x7f02880d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39128/0 (socket says 172.21.15.33:39128) 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f0285d9b640 1 -- 172.21.15.33:0/2970854741 learned_addr learned my addr 172.21.15.33:0/2970854741 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f0286d9d640 1 --2- 172.21.15.33:0/2970854741 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d2ac0 0x7f0288167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028659c640 1 -- 172.21.15.33:0/2970854741 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d2ac0 msgr2=0x7f0288167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028659c640 1 --2- 172.21.15.33:0/2970854741 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d2ac0 0x7f0288167cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028659c640 1 -- 172.21.15.33:0/2970854741 >> v1:172.21.15.33:6789/0 conn(0x7f02880d1f60 legacy=0x7f02880d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028559a640 1 -- 172.21.15.33:0/2970854741 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2308720838 0 0) 0x7f02880b6cc0 con 0x7f02880d1f60 2024-04-25T01:09:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.289+0000 7f028659c640 1 -- 172.21.15.33:0/2970854741 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02880b7210 con 0x7f028815e0d0 2024-04-25T01:09:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028659c640 1 --2- 172.21.15.33:0/2970854741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880cf3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f02780097c0 tx=0x7f027801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2daf20fbd44922 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028559a640 1 -- 172.21.15.33:0/2970854741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02780b1070 con 0x7f028815e0d0 2024-04-25T01:09:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028559a640 1 -- 172.21.15.33:0/2970854741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02780ac630 con 0x7f028815e0d0 2024-04-25T01:09:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028559a640 1 -- 172.21.15.33:0/2970854741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02780a8050 con 0x7f028815e0d0 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 -- 172.21.15.33:0/2970854741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 msgr2=0x7f02880cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 --2- 172.21.15.33:0/2970854741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880cf3f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f02780097c0 tx=0x7f027801c4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 -- 172.21.15.33:0/2970854741 shutdown_connections 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 --2- 172.21.15.33:0/2970854741 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d2ac0 0x7f0288167cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 --2- 172.21.15.33:0/2970854741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880cf3f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 -- 172.21.15.33:0/2970854741 >> 172.21.15.33:0/2970854741 conn(0x7f02880d3dd0 msgr2=0x7f02880d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 -- 172.21.15.33:0/2970854741 shutdown_connections 2024-04-25T01:09:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.293+0000 7f028c9a2640 1 -- 172.21.15.33:0/2970854741 wait complete. 2024-04-25T01:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 Processor -- start 2024-04-25T01:09:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 -- start start 2024-04-25T01:09:07.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d1f60 0x7f02880c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02880d2ac0 0x7f02880b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f028816abd0 con 0x7f028815e0d0 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f028816aa50 con 0x7f02880d2ac0 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f028816ad50 con 0x7f02880d1f60 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880b7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51446/0 (socket says 172.21.15.33:51446) 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 -- 172.21.15.33:0/1076170691 learned_addr learned my addr 172.21.15.33:0/1076170691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028659c640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d1f60 0x7f02880c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0285d9b640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02880d2ac0 0x7f02880b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 -- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d1f60 msgr2=0x7f02880c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d1f60 0x7f02880c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 -- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02880d2ac0 msgr2=0x7f02880b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02880d2ac0 0x7f02880b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 -- 172.21.15.33:0/1076170691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02880c8fc0 con 0x7f028815e0d0 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0286d9d640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880b7f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0280007e30 tx=0x7f028000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0276ffd640 1 -- 172.21.15.33:0/1076170691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280014620 con 0x7f028815e0d0 2024-04-25T01:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0276ffd640 1 -- 172.21.15.33:0/1076170691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0280019070 con 0x7f028815e0d0 2024-04-25T01:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02880c92a0 con 0x7f028815e0d0 2024-04-25T01:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0276ffd640 1 -- 172.21.15.33:0/1076170691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280014a00 con 0x7f028815e0d0 2024-04-25T01:09:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02881f6dd0 con 0x7f028815e0d0 2024-04-25T01:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0288069040 con 0x7f028815e0d0 2024-04-25T01:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.297+0000 7f0276ffd640 1 -- 172.21.15.33:0/1076170691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0280014ba0 con 0x7f028815e0d0 2024-04-25T01:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.301+0000 7f0276ffd640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f02680678c0 0x7f0268069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.301+0000 7f0276ffd640 1 -- 172.21.15.33:0/1076170691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f02800868c0 con 0x7f028815e0d0 2024-04-25T01:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.301+0000 7f028659c640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f02680678c0 0x7f0268069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.301+0000 7f028659c640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f02680678c0 0x7f0268069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f02780ae000 tx=0x7f0278008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.305+0000 7f0276ffd640 1 -- 172.21.15.33:0/1076170691 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f02800570c0 con 0x7f028815e0d0 2024-04-25T01:09:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.461+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0288167c80 con 0x7f028815e0d0 2024-04-25T01:09:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.461+0000 7f0276ffd640 1 -- 172.21.15.33:0/1076170691 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f02800569c0 con 0x7f028815e0d0 2024-04-25T01:09:07.455 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:07.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f02680678c0 msgr2=0x7f0268069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f02680678c0 0x7f0268069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f02780ae000 tx=0x7f0278008040 comp rx=0 tx=0).stop 2024-04-25T01:09:07.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 msgr2=0x7f02880b7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880b7f50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0280007e30 tx=0x7f028000e4b0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 shutdown_connections 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f02680678c0 0x7f0268069d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f028815e0d0 0x7f02880b7f50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02880d2ac0 0x7f02880b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 --2- 172.21.15.33:0/1076170691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f02880d1f60 0x7f02880c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 >> 172.21.15.33:0/1076170691 conn(0x7f02880d3dd0 msgr2=0x7f02880d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 shutdown_connections 2024-04-25T01:09:07.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.465+0000 7f028c9a2640 1 -- 172.21.15.33:0/1076170691 wait complete. 2024-04-25T01:09:07.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:07.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6de538640 1 Processor -- start 2024-04-25T01:09:07.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6de538640 1 -- start start 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6de538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 0x7fd6d80ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6de538640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d815e480 con 0x7fd6d80aec20 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6de538640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6d806df00 con 0x7fd6d815e0a0 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6de538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d80ae220 con 0x7fd6d80716a0 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6dcad5640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd6d80aec20 0x7fd6d806cd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39130/0 (socket says 172.21.15.33:39130) 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6dcad5640 1 -- 172.21.15.33:0/1566694804 learned_addr learned my addr 172.21.15.33:0/1566694804 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d77fe640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 0x7fd6d80ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252722380 0 0) 0x7fd6d815e480 con 0x7fd6d80aec20 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6c0003530 con 0x7fd6d80aec20 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2272773388 0 0) 0x7fd6c0003530 con 0x7fd6d80aec20 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 msgr2=0x7fd6d80ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:07.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 0x7fd6d80ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 >> v1:172.21.15.138:6789/0 conn(0x7fd6d815e0a0 legacy=0x7fd6d80b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d8071b00 con 0x7fd6d80aec20 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fd6d0001680 con 0x7fd6d80aec20 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 >> v1:172.21.15.33:6789/0 conn(0x7fd6d80aec20 legacy=0x7fd6d806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6c00039f0 0x7fd6c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6c0004810 0x7fd6c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 0x7fd6c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d80ae220 con 0x7fd6c00039f0 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c000bb00 con 0x7fd6c0004810 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c000bd00 con 0x7fd6d80716a0 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d7fff640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6c00039f0 0x7fd6c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d77fe640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6c0004810 0x7fd6c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d7fff640 1 -- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 msgr2=0x7fd6c0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:07.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6dcad5640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 0x7fd6c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d7fff640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80716a0 0x7fd6c0005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d7fff640 1 -- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6c0004810 msgr2=0x7fd6c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d7fff640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6c0004810 0x7fd6c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d7fff640 1 -- 172.21.15.33:0/1566694804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c0003530 con 0x7fd6c00039f0 2024-04-25T01:09:07.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.597+0000 7fd6d77fe640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6c0004810 0x7fd6c0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:07.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6d7fff640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6c00039f0 0x7fd6c0003e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8001e70 tx=0x7fd6c801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726726155a4fc04f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80af070 con 0x7fd6c00039f0 2024-04-25T01:09:07.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6c8003b80 con 0x7fd6c00039f0 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6d6ffd640 1 -- 172.21.15.33:0/1566694804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8001a90 con 0x7fd6c00039f0 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 -- 172.21.15.33:0/1566694804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6c00039f0 msgr2=0x7fd6c0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 --2- 172.21.15.33:0/1566694804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6c00039f0 0x7fd6c0003e10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8001e70 tx=0x7fd6c801c780 comp rx=0 tx=0).stop 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6dcad5640 1 -- 172.21.15.33:0/1566694804 reap_dead start 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 -- 172.21.15.33:0/1566694804 shutdown_connections 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 -- 172.21.15.33:0/1566694804 >> 172.21.15.33:0/1566694804 conn(0x7fd6d80b1810 msgr2=0x7fd6d80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 -- 172.21.15.33:0/1566694804 shutdown_connections 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 -- 172.21.15.33:0/1566694804 wait complete. 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 Processor -- start 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.601+0000 7fd6de538640 1 -- start start 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80ce5d0 0x7fd6d80ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cc4f0 0x7fd6d8160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80cdb00 0x7fd6d8156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d80af1d0 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d80af050 con 0x7fd6d80ce5d0 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd6d8169f70 con 0x7fd6d80cdb00 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cc4f0 0x7fd6d8160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cc4f0 0x7fd6d8160e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51454/0 (socket says 172.21.15.33:51454) 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1243886790 learned_addr learned my addr 172.21.15.33:0/1243886790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6dcad5640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80cdb00 0x7fd6d8156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d7fff640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80ce5d0 0x7fd6d80ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80cdb00 msgr2=0x7fd6d8156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80cdb00 0x7fd6d8156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80ce5d0 msgr2=0x7fd6d80ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80ce5d0 0x7fd6d80ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1243886790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d80ceef0 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d77fe640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cc4f0 0x7fd6d8160e50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0108c0 tx=0x7fd6cc010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d4ff9640 1 -- 172.21.15.33:0/1243886790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00f040 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d4ff9640 1 -- 172.21.15.33:0/1243886790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6cc004220 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d80cca60 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d4ff9640 1 -- 172.21.15.33:0/1243886790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00e900 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d80cd020 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d8068c60 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d4ff9640 1 -- 172.21.15.33:0/1243886790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd6cc00eaa0 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d4ff9640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6c40678f0 0x7fd6c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d4ff9640 1 -- 172.21.15.33:0/1243886790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd6cc0861c0 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.605+0000 7fd6d7fff640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6c40678f0 0x7fd6c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.609+0000 7fd6d7fff640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6c40678f0 0x7fd6c4069db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80ac000 tx=0x7fd6c8001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.609+0000 7fd6d4ff9640 1 -- 172.21.15.33:0/1243886790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd6cc018030 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.769+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6d815d8d0 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.769+0000 7fd6d4ff9640 1 -- 172.21.15.33:0/1243886790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fd6cc056930 con 0x7fd6d80cc4f0 2024-04-25T01:09:07.763 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6c40678f0 msgr2=0x7fd6c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6c40678f0 0x7fd6c4069db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd6c80ac000 tx=0x7fd6c8001520 comp rx=0 tx=0).stop 2024-04-25T01:09:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cc4f0 msgr2=0x7fd6d8160e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cc4f0 0x7fd6d8160e50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc0108c0 tx=0x7fd6cc010d80 comp rx=0 tx=0).stop 2024-04-25T01:09:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 shutdown_connections 2024-04-25T01:09:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd6c40678f0 0x7fd6c4069db0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd6d80cdb00 0x7fd6d8156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d80cc4f0 0x7fd6d8160e50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 --2- 172.21.15.33:0/1243886790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80ce5d0 0x7fd6d80ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 >> 172.21.15.33:0/1243886790 conn(0x7fd6d80b1810 msgr2=0x7fd6d8159180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 shutdown_connections 2024-04-25T01:09:07.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.773+0000 7fd6de538640 1 -- 172.21.15.33:0/1243886790 wait complete. 2024-04-25T01:09:07.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b4691b640 1 Processor -- start 2024-04-25T01:09:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b4691b640 1 -- start start 2024-04-25T01:09:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b4691b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3b4015e0a0 0x7f3b400716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b4691b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4015e480 con 0x7f3b400b31a0 2024-04-25T01:09:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b4691b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4006df00 con 0x7f3b400aedf0 2024-04-25T01:09:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b4691b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3b400ae220 con 0x7f3b4015e0a0 2024-04-25T01:09:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3f7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f3b400b31a0 0x7f3b400ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39140/0 (socket says 172.21.15.33:39140) 2024-04-25T01:09:07.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3f7fe640 1 -- 172.21.15.33:0/1163110059 learned_addr learned my addr 172.21.15.33:0/1163110059 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1163110059 >> v2:172.21.15.138:3300/0 conn(0x7f3b4015e0a0 0x7f3b400716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073400945 0 0) 0x7f3b4015e480 con 0x7f3b400b31a0 2024-04-25T01:09:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b28003530 con 0x7f3b400b31a0 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 785970521 0 0) 0x7f3b28003530 con 0x7f3b400b31a0 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 >> v2:172.21.15.138:3300/0 conn(0x7f3b4015e0a0 msgr2=0x7f3b400716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 --2- 172.21.15.33:0/1163110059 >> v2:172.21.15.138:3300/0 conn(0x7f3b4015e0a0 0x7f3b400716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 >> v1:172.21.15.138:6789/0 conn(0x7f3b400aedf0 legacy=0x7f3b4006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4006e880 con 0x7f3b400b31a0 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f3b34001680 con 0x7f3b400b31a0 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 >> v1:172.21.15.33:6789/0 conn(0x7f3b400b31a0 legacy=0x7f3b400ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b28003a10 0x7f3b28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b28004810 0x7f3b28004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b2800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400ae220 con 0x7f3b28005220 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2800c170 con 0x7f3b28003a10 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2800c370 con 0x7f3b28004810 2024-04-25T01:09:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b44eb8640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b2800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b28003a10 0x7f3b28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b3f7fe640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b28004810 0x7f3b28004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b44eb8640 1 -- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b28004810 msgr2=0x7f3b28004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b44eb8640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b28004810 0x7f3b28004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b44eb8640 1 -- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b28003a10 msgr2=0x7f3b28003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b44eb8640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b28003a10 0x7f3b28003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.909+0000 7f3b44eb8640 1 -- 172.21.15.33:0/1163110059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b28003530 con 0x7f3b28005220 2024-04-25T01:09:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b44eb8640 1 -- 172.21.15.33:0/1163110059 reap_dead start 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b44eb8640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b2800b9d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3b38001e40 tx=0x7f3b3801c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623cc937808b3d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38001a90 con 0x7f3b28005220 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b38003b80 con 0x7f3b28005220 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b3effd640 1 -- 172.21.15.33:0/1163110059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38005020 con 0x7f3b28005220 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- 172.21.15.33:0/1163110059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 msgr2=0x7f3b2800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b2800b9d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3b38001e40 tx=0x7f3b3801c720 comp rx=0 tx=0).stop 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- 172.21.15.33:0/1163110059 shutdown_connections 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 --2- 172.21.15.33:0/1163110059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b2800b9d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- 172.21.15.33:0/1163110059 >> 172.21.15.33:0/1163110059 conn(0x7f3b400b1810 msgr2=0x7f3b400b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- 172.21.15.33:0/1163110059 shutdown_connections 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- 172.21.15.33:0/1163110059 wait complete. 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 Processor -- start 2024-04-25T01:09:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- start start 2024-04-25T01:09:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b400c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b4014c680 0x7f3b4015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b4015d0c0 0x7f3b4015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400b3630 con 0x7f3b28005220 2024-04-25T01:09:07.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4006eb60 con 0x7f3b4014c680 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b4691b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400b37b0 con 0x7f3b4015d0c0 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b400c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b3ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b400c4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51462/0 (socket says 172.21.15.33:51462) 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.913+0000 7f3b3ffff640 1 -- 172.21.15.33:0/264943199 learned_addr learned my addr 172.21.15.33:0/264943199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3f7fe640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b4014c680 0x7f3b4015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b44eb8640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b4015d0c0 0x7f3b4015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3ffff640 1 -- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b4015d0c0 msgr2=0x7f3b4015d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b4015d0c0 0x7f3b4015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3ffff640 1 -- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b4014c680 msgr2=0x7f3b4015b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b4014c680 0x7f3b4015b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3ffff640 1 -- 172.21.15.33:0/264943199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4015bc60 con 0x7f3b28005220 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3ffff640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b400c4c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3b30011590 tx=0x7f3b30011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3cff9640 1 -- 172.21.15.33:0/264943199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30015930 con 0x7f3b28005220 2024-04-25T01:09:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3cff9640 1 -- 172.21.15.33:0/264943199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b30015ad0 con 0x7f3b28005220 2024-04-25T01:09:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4015c300 con 0x7f3b28005220 2024-04-25T01:09:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3cff9640 1 -- 172.21.15.33:0/264943199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3001d400 con 0x7f3b28005220 2024-04-25T01:09:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4015c8c0 con 0x7f3b28005220 2024-04-25T01:09:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b40068c60 con 0x7f3b28005220 2024-04-25T01:09:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3cff9640 1 -- 172.21.15.33:0/264943199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b3001d5a0 con 0x7f3b28005220 2024-04-25T01:09:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3cff9640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3b2c0678f0 0x7f3b2c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3cff9640 1 -- 172.21.15.33:0/264943199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3b300867a0 con 0x7f3b28005220 2024-04-25T01:09:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3f7fe640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3b2c0678f0 0x7f3b2c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.917+0000 7f3b3f7fe640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3b2c0678f0 0x7f3b2c069db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b40159610 tx=0x7f3b34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:07.921+0000 7f3b3cff9640 1 -- 172.21.15.33:0/264943199 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3b4015bc60 con 0x7f3b28005220 2024-04-25T01:09:08.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.073+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3b40069040 con 0x7f3b28005220 2024-04-25T01:09:08.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.073+0000 7f3b3cff9640 1 -- 172.21.15.33:0/264943199 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f3b30019030 con 0x7f3b28005220 2024-04-25T01:09:08.067 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:08.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3b2c0678f0 msgr2=0x7f3b2c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3b2c0678f0 0x7f3b2c069db0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b40159610 tx=0x7f3b34009040 comp rx=0 tx=0).stop 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 msgr2=0x7f3b400c4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b400c4c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3b30011590 tx=0x7f3b30011a50 comp rx=0 tx=0).stop 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 shutdown_connections 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3b2c0678f0 0x7f3b2c069db0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3b4015d0c0 0x7f3b4015d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b4014c680 0x7f3b4015b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 --2- 172.21.15.33:0/264943199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b28005220 0x7f3b400c4c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 >> 172.21.15.33:0/264943199 conn(0x7f3b400b1810 msgr2=0x7f3b400b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 shutdown_connections 2024-04-25T01:09:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.077+0000 7f3b4691b640 1 -- 172.21.15.33:0/264943199 wait complete. 2024-04-25T01:09:08.085 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe621884640 1 Processor -- start 2024-04-25T01:09:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe621884640 1 -- start start 2024-04-25T01:09:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe621884640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe61c15e0a0 0x7fe61c0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe621884640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 0x7fe61c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe621884640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe61c15e480 con 0x7fe61c0b31a0 2024-04-25T01:09:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe621884640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe61c0ae220 con 0x7fe61c15e0a0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe621884640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c06e880 con 0x7fe61c0aedf0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61a7fc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe61c0b31a0 0x7fe61c0ae6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39152/0 (socket says 172.21.15.33:39152) 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61a7fc640 1 -- 172.21.15.33:0/1429099584 learned_addr learned my addr 172.21.15.33:0/1429099584 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61affd640 1 --2- 172.21.15.33:0/1429099584 >> v2:172.21.15.138:3300/0 conn(0x7fe61c15e0a0 0x7fe61c0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 0x7fe61c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2563588180 0 0) 0x7fe61c15e480 con 0x7fe61c0b31a0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe608003530 con 0x7fe61c0b31a0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1914817759 0 0) 0x7fe608003530 con 0x7fe61c0b31a0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 msgr2=0x7fe61c06cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 0x7fe61c06cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 >> v2:172.21.15.138:3300/0 conn(0x7fe61c15e0a0 msgr2=0x7fe61c0716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 --2- 172.21.15.33:0/1429099584 >> v2:172.21.15.138:3300/0 conn(0x7fe61c15e0a0 0x7fe61c0716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c06df00 con 0x7fe61c0b31a0 2024-04-25T01:09:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fe604001680 con 0x7fe61c0b31a0 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 >> v1:172.21.15.33:6789/0 conn(0x7fe61c0b31a0 legacy=0x7fe61c0ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6080039c0 0x7fe608003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 0x7fe608004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608004d50 0x7fe6080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0ae220 con 0x7fe608004d50 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c06e880 con 0x7fe6080039c0 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe60800ba00 con 0x7fe61c0aedf0 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608004d50 0x7fe6080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61affd640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6080039c0 0x7fe608003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61a7fc640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 0x7fe608004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61b7fe640 1 -- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 msgr2=0x7fe608004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c0aedf0 0x7fe608004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61b7fe640 1 -- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6080039c0 msgr2=0x7fe608003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6080039c0 0x7fe608003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.213+0000 7fe61b7fe640 1 -- 172.21.15.33:0/1429099584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe608003530 con 0x7fe608004d50 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608004d50 0x7fe6080051d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe614009fa0 tx=0x7fe61401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a7e442225780fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140b2070 con 0x7fe608004d50 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6140a5c00 con 0x7fe608004d50 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe619ffb640 1 -- 172.21.15.33:0/1429099584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614009bf0 con 0x7fe608004d50 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 -- 172.21.15.33:0/1429099584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608004d50 msgr2=0x7fe6080051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 --2- 172.21.15.33:0/1429099584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608004d50 0x7fe6080051d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe614009fa0 tx=0x7fe61401cc40 comp rx=0 tx=0).stop 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe61b7fe640 1 -- 172.21.15.33:0/1429099584 reap_dead start 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 -- 172.21.15.33:0/1429099584 shutdown_connections 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 -- 172.21.15.33:0/1429099584 >> 172.21.15.33:0/1429099584 conn(0x7fe61c0b1810 msgr2=0x7fe61c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 -- 172.21.15.33:0/1429099584 shutdown_connections 2024-04-25T01:09:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 -- 172.21.15.33:0/1429099584 wait complete. 2024-04-25T01:09:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 Processor -- start 2024-04-25T01:09:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.217+0000 7fe621884640 1 -- start start 2024-04-25T01:09:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe608004d50 0x7fe61c15b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c157320 0x7fe61c1577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe61c159d40 0x7fe61c15a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0b3630 con 0x7fe61c159d40 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c06eb60 con 0x7fe608004d50 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c0b37b0 con 0x7fe61c157320 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe61c159d40 0x7fe61c15a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe61c159d40 0x7fe61c15a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51492/0 (socket says 172.21.15.33:51492) 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 -- 172.21.15.33:0/620678181 learned_addr learned my addr 172.21.15.33:0/620678181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61a7fc640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c157320 0x7fe61c1577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61affd640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe608004d50 0x7fe61c15b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 -- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c157320 msgr2=0x7fe61c1577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c157320 0x7fe61c1577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 -- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe608004d50 msgr2=0x7fe61c15b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe608004d50 0x7fe61c15b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 -- 172.21.15.33:0/620678181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c0cc4f0 con 0x7fe61c159d40 2024-04-25T01:09:08.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61b7fe640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe61c159d40 0x7fe61c15a1c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe614002240 tx=0x7fe614009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe613fff640 1 -- 172.21.15.33:0/620678181 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140b2040 con 0x7fe61c159d40 2024-04-25T01:09:08.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe613fff640 1 -- 172.21.15.33:0/620678181 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6140a5a00 con 0x7fe61c159d40 2024-04-25T01:09:08.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c15a920 con 0x7fe61c159d40 2024-04-25T01:09:08.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe613fff640 1 -- 172.21.15.33:0/620678181 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614009bf0 con 0x7fe61c159d40 2024-04-25T01:09:08.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c15aee0 con 0x7fe61c159d40 2024-04-25T01:09:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c068c60 con 0x7fe61c159d40 2024-04-25T01:09:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe613fff640 1 -- 172.21.15.33:0/620678181 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe6140c0030 con 0x7fe61c159d40 2024-04-25T01:09:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe613fff640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe5fc0678c0 0x7fe5fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe613fff640 1 -- 172.21.15.33:0/620678181 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe6141207f0 con 0x7fe61c159d40 2024-04-25T01:09:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.221+0000 7fe61affd640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe5fc0678c0 0x7fe5fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.225+0000 7fe61affd640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe5fc0678c0 0x7fe5fc069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009a30 tx=0x7fe60c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.225+0000 7fe613fff640 1 -- 172.21.15.33:0/620678181 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe6140c7060 con 0x7fe61c159d40 2024-04-25T01:09:08.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.381+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe61c069040 con 0x7fe61c159d40 2024-04-25T01:09:08.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.381+0000 7fe613fff640 1 -- 172.21.15.33:0/620678181 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fe614016950 con 0x7fe61c159d40 2024-04-25T01:09:08.375 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe5fc0678c0 msgr2=0x7fe5fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe5fc0678c0 0x7fe5fc069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009a30 tx=0x7fe60c008040 comp rx=0 tx=0).stop 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe61c159d40 msgr2=0x7fe61c15a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe61c159d40 0x7fe61c15a1c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe614002240 tx=0x7fe614009750 comp rx=0 tx=0).stop 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 shutdown_connections 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe5fc0678c0 0x7fe5fc069d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe61c159d40 0x7fe61c15a1c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe61c157320 0x7fe61c1577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 --2- 172.21.15.33:0/620678181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe608004d50 0x7fe61c15b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 >> 172.21.15.33:0/620678181 conn(0x7fe61c0b1810 msgr2=0x7fe61c06cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:08.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 shutdown_connections 2024-04-25T01:09:08.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.385+0000 7fe621884640 1 -- 172.21.15.33:0/620678181 wait complete. 2024-04-25T01:09:08.393 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:08.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafae065640 1 Processor -- start 2024-04-25T01:09:08.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafae065640 1 -- start start 2024-04-25T01:09:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafae065640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fafa815c160 0x7fafa80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafae065640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 0x7fafa80cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafae065640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa815f9a0 con 0x7fafa8145470 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafae065640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fafa80b57b0 con 0x7fafa815c160 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafae065640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fafa815f750 con 0x7fafa80b5360 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa77fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fafa8145470 0x7fafa80cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39160/0 (socket says 172.21.15.33:39160) 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa77fe640 1 -- 172.21.15.33:0/3959660998 learned_addr learned my addr 172.21.15.33:0/3959660998 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa7fff640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 0x7fafa80cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa6ffd640 1 --2- 172.21.15.33:0/3959660998 >> v2:172.21.15.138:3300/0 conn(0x7fafa815c160 0x7fafa80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3177297552 0 0) 0x7fafa815f9a0 con 0x7fafa8145470 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf94003530 con 0x7fafa8145470 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 881109389 0 0) 0x7faf94003530 con 0x7fafa8145470 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 msgr2=0x7fafa80cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 0x7fafa80cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 >> v2:172.21.15.138:3300/0 conn(0x7fafa815c160 msgr2=0x7fafa80b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 --2- 172.21.15.33:0/3959660998 >> v2:172.21.15.138:3300/0 conn(0x7fafa815c160 0x7fafa80b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa815fc80 con 0x7fafa8145470 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7faf980016a0 con 0x7fafa8145470 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 >> v1:172.21.15.33:6789/0 conn(0x7fafa8145470 legacy=0x7fafa80cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf940039c0 0x7faf94003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 0x7faf94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94004d50 0x7faf940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fafa80b57b0 con 0x7faf94004d50 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafa815f750 con 0x7faf940039c0 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faf9400b9d0 con 0x7fafa80b5360 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa7fff640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94004d50 0x7faf940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa77fe640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf940039c0 0x7faf94003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa7fff640 1 -- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 msgr2=0x7faf94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa6ffd640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 0x7faf94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa7fff640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa80b5360 0x7faf94004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa7fff640 1 -- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf940039c0 msgr2=0x7faf94003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa7fff640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf940039c0 0x7faf94003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.513+0000 7fafa7fff640 1 -- 172.21.15.33:0/3959660998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf94003530 con 0x7faf94004d50 2024-04-25T01:09:08.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94004d50 0x7faf940051d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009b10 tx=0x7fafa001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=457e2462a31639da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00b2070 con 0x7faf94004d50 2024-04-25T01:09:08.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa00a5660 con 0x7faf94004d50 2024-04-25T01:09:08.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa67fc640 1 -- 172.21.15.33:0/3959660998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00096b0 con 0x7faf94004d50 2024-04-25T01:09:08.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- 172.21.15.33:0/3959660998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94004d50 msgr2=0x7faf940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 --2- 172.21.15.33:0/3959660998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf94004d50 0x7faf940051d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009b10 tx=0x7fafa001ca70 comp rx=0 tx=0).stop 2024-04-25T01:09:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 -- 172.21.15.33:0/3959660998 reap_dead start 2024-04-25T01:09:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- 172.21.15.33:0/3959660998 shutdown_connections 2024-04-25T01:09:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- 172.21.15.33:0/3959660998 >> 172.21.15.33:0/3959660998 conn(0x7fafa80b1810 msgr2=0x7fafa80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- 172.21.15.33:0/3959660998 shutdown_connections 2024-04-25T01:09:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- 172.21.15.33:0/3959660998 wait complete. 2024-04-25T01:09:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 Processor -- start 2024-04-25T01:09:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- start start 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa815b680 0x7fafa81f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa815e5e0 0x7fafa813f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa813fa00 0x7fafa81f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fafa80cf7f0 con 0x7fafa813fa00 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fafa8160240 con 0x7fafa815b680 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafae065640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fafa8145850 con 0x7fafa815e5e0 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa813fa00 0x7fafa81f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa813fa00 0x7fafa81f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51514/0 (socket says 172.21.15.33:51514) 2024-04-25T01:09:08.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 -- 172.21.15.33:0/1318602608 learned_addr learned my addr 172.21.15.33:0/1318602608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa6ffd640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa815e5e0 0x7fafa813f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa77fe640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa815b680 0x7fafa81f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 -- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa815e5e0 msgr2=0x7fafa813f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa815e5e0 0x7fafa813f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 -- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa815b680 msgr2=0x7fafa81f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa815b680 0x7fafa81f28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa7fff640 1 -- 172.21.15.33:0/1318602608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa815c540 con 0x7fafa813fa00 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.517+0000 7fafa6ffd640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa815e5e0 0x7fafa813f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7fafa7fff640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa813fa00 0x7fafa81f1a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009b10 tx=0x7faf94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7faf9ffff640 1 -- 172.21.15.33:0/1318602608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00bb070 con 0x7fafa813fa00 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7faf9ffff640 1 -- 172.21.15.33:0/1318602608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa0000e50 con 0x7fafa813fa00 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7faf9ffff640 1 -- 172.21.15.33:0/1318602608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00b2040 con 0x7fafa813fa00 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa815c7e0 con 0x7fafa813fa00 2024-04-25T01:09:08.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa815cda0 con 0x7fafa813fa00 2024-04-25T01:09:08.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa8069040 con 0x7fafa813fa00 2024-04-25T01:09:08.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7faf9ffff640 1 -- 172.21.15.33:0/1318602608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fafa00be070 con 0x7fafa813fa00 2024-04-25T01:09:08.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7faf9ffff640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf880678f0 0x7faf88069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7faf9ffff640 1 -- 172.21.15.33:0/1318602608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fafa011f260 con 0x7fafa813fa00 2024-04-25T01:09:08.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7fafa77fe640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf880678f0 0x7faf88069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.521+0000 7fafa77fe640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf880678f0 0x7faf88069db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fafa80af750 tx=0x7faf98000f60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.525+0000 7faf9ffff640 1 -- 172.21.15.33:0/1318602608 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fafa00167c0 con 0x7fafa813fa00 2024-04-25T01:09:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.684+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafa81f1780 con 0x7fafa813fa00 2024-04-25T01:09:08.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.684+0000 7faf9ffff640 1 -- 172.21.15.33:0/1318602608 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7fafa00ef410 con 0x7fafa813fa00 2024-04-25T01:09:08.678 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:08.679 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf880678f0 msgr2=0x7faf88069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf880678f0 0x7faf88069db0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fafa80af750 tx=0x7faf98000f60 comp rx=0 tx=0).stop 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa813fa00 msgr2=0x7fafa81f1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa813fa00 0x7fafa81f1a20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009b10 tx=0x7faf94005840 comp rx=0 tx=0).stop 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 shutdown_connections 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf880678f0 0x7faf88069db0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafa813fa00 0x7fafa81f1a20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fafa815e5e0 0x7fafa813f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 --2- 172.21.15.33:0/1318602608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafa815b680 0x7fafa81f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 >> 172.21.15.33:0/1318602608 conn(0x7fafa80b1810 msgr2=0x7fafa80cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 shutdown_connections 2024-04-25T01:09:08.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.688+0000 7fafae065640 1 -- 172.21.15.33:0/1318602608 wait complete. 2024-04-25T01:09:08.697 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:08.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:08.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 Processor -- start 2024-04-25T01:09:08.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 -- start start 2024-04-25T01:09:08.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e5806b210 0x7f2e580b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58067030 0x7f2e5806b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2e58072910 0x7f2e58072d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f2e580b61b0 con 0x7f2e5806b210 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2e5806bb30 con 0x7f2e58072910 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5eeb6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e580b55a0 con 0x7f2e58067030 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e5806b210 0x7f2e580b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2e5806b210 0x7f2e580b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46620/0 (socket says 172.21.15.33:46620) 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 -- 172.21.15.33:0/3267285187 learned_addr learned my addr 172.21.15.33:0/3267285187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e57fff640 1 --2- 172.21.15.33:0/3267285187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58067030 0x7f2e5806b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5d453640 1 --2- 172.21.15.33:0/3267285187 >> v2:172.21.15.138:3300/0 conn(0x7f2e58072910 0x7f2e58072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 -- 172.21.15.33:0/3267285187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58067030 msgr2=0x7f2e5806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 --2- 172.21.15.33:0/3267285187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58067030 0x7f2e5806b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 -- 172.21.15.33:0/3267285187 >> v2:172.21.15.138:3300/0 conn(0x7f2e58072910 msgr2=0x7f2e58072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 --2- 172.21.15.33:0/3267285187 >> v2:172.21.15.138:3300/0 conn(0x7f2e58072910 0x7f2e58072d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5cc52640 1 -- 172.21.15.33:0/3267285187 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e580b5880 con 0x7f2e5806b210 2024-04-25T01:09:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e57fff640 1 --2- 172.21.15.33:0/3267285187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58067030 0x7f2e5806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:08.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.820+0000 7f2e5d453640 1 --2- 172.21.15.33:0/3267285187 >> v2:172.21.15.138:3300/0 conn(0x7f2e58072910 0x7f2e58072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5cc52640 1 --2- 172.21.15.33:0/3267285187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5806b210 0x7f2e580b4fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2e58072320 tx=0x7f2e4801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b5c15ce87ff7ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e577fe640 1 -- 172.21.15.33:0/3267285187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480ab420 con 0x7f2e5806b210 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e577fe640 1 -- 172.21.15.33:0/3267285187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e480b0070 con 0x7f2e5806b210 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e577fe640 1 -- 172.21.15.33:0/3267285187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480ab7e0 con 0x7f2e5806b210 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/3267285187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5806b210 msgr2=0x7f2e580b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/3267285187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5806b210 0x7f2e580b4fc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2e58072320 tx=0x7f2e4801c460 comp rx=0 tx=0).stop 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/3267285187 shutdown_connections 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/3267285187 >> v2:172.21.15.138:3300/0 conn(0x7f2e58072910 0x7f2e58072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/3267285187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58067030 0x7f2e5806b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/3267285187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e5806b210 0x7f2e580b4fc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/3267285187 >> 172.21.15.33:0/3267285187 conn(0x7f2e580b1e60 msgr2=0x7f2e580b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:08.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/3267285187 shutdown_connections 2024-04-25T01:09:08.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/3267285187 wait complete. 2024-04-25T01:09:08.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 Processor -- start 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.824+0000 7f2e5eeb6640 1 -- start start 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58067030 0x7f2e58167930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5806b210 0x7f2e580ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58072910 0x7f2e5815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580cc590 con 0x7f2e58067030 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e580ca8e0 con 0x7f2e5806b210 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2e580cc710 con 0x7f2e58072910 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58067030 0x7f2e58167930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58067030 0x7f2e58167930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46632/0 (socket says 172.21.15.33:46632) 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 -- 172.21.15.33:0/1868613936 learned_addr learned my addr 172.21.15.33:0/1868613936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5d453640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58072910 0x7f2e5815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 -- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58072910 msgr2=0x7f2e5815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58072910 0x7f2e5815dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 -- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5806b210 msgr2=0x7f2e580ae390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5806b210 0x7f2e580ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 -- 172.21.15.33:0/1868613936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5815f550 con 0x7f2e58067030 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5cc52640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58067030 0x7f2e58167930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801ab10 tx=0x7f2e4801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e557fa640 1 -- 172.21.15.33:0/1868613936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480abda0 con 0x7f2e58067030 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e557fa640 1 -- 172.21.15.33:0/1868613936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e480be3f0 con 0x7f2e58067030 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e5815f830 con 0x7f2e58067030 2024-04-25T01:09:08.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e557fa640 1 -- 172.21.15.33:0/1868613936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480be7e0 con 0x7f2e58067030 2024-04-25T01:09:08.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e5815f130 con 0x7f2e58067030 2024-04-25T01:09:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e580c7830 con 0x7f2e58067030 2024-04-25T01:09:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e557fa640 1 -- 172.21.15.33:0/1868613936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e480b43f0 con 0x7f2e58067030 2024-04-25T01:09:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e557fa640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2e380678f0 0x7f2e38069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e557fa640 1 -- 172.21.15.33:0/1868613936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2e481202e0 con 0x7f2e58067030 2024-04-25T01:09:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e57fff640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2e380678f0 0x7f2e38069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:08.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.828+0000 7f2e57fff640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2e380678f0 0x7f2e38069db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e40006b20 tx=0x7f2e40008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:08.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.832+0000 7f2e557fa640 1 -- 172.21.15.33:0/1868613936 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2e480c4060 con 0x7f2e58067030 2024-04-25T01:09:08.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.992+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2e58069040 con 0x7f2e58067030 2024-04-25T01:09:08.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.992+0000 7f2e557fa640 1 -- 172.21.15.33:0/1868613936 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f2e48016110 con 0x7f2e58067030 2024-04-25T01:09:08.987 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:08.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2e380678f0 msgr2=0x7f2e38069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2e380678f0 0x7f2e38069db0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e40006b20 tx=0x7f2e40008040 comp rx=0 tx=0).stop 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58067030 msgr2=0x7f2e58167930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58067030 0x7f2e58167930 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801ab10 tx=0x7f2e4801aca0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 shutdown_connections 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2e380678f0 0x7f2e38069db0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2e58072910 0x7f2e5815dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e5806b210 0x7f2e580ae390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 --2- 172.21.15.33:0/1868613936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e58067030 0x7f2e58167930 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 >> 172.21.15.33:0/1868613936 conn(0x7f2e580b1e60 msgr2=0x7f2e580caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 shutdown_connections 2024-04-25T01:09:08.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:08.996+0000 7f2e5eeb6640 1 -- 172.21.15.33:0/1868613936 wait complete. 2024-04-25T01:09:09.006 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7967810640 1 Processor -- start 2024-04-25T01:09:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7967810640 1 -- start start 2024-04-25T01:09:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7967810640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f79600716a0 0x7f79600ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7967810640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79600aec20 0x7f796006cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7967810640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f796015e480 con 0x7f796015e0a0 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7967810640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f7960071b30 con 0x7f79600aec20 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7967810640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f79600ae220 con 0x7f79600716a0 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7965dad640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79600aec20 0x7f796006cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7965dad640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79600aec20 0x7f796006cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46644/0 (socket says 172.21.15.33:46644) 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.136+0000 7f7965dad640 1 -- 172.21.15.33:0/3905509384 learned_addr learned my addr 172.21.15.33:0/3905509384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7964dab640 1 --2- 172.21.15.33:0/3905509384 >> v2:172.21.15.138:3300/0 conn(0x7f79600716a0 0x7f79600ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7965dad640 1 -- 172.21.15.33:0/3905509384 >> v2:172.21.15.138:3300/0 conn(0x7f79600716a0 msgr2=0x7f79600ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7965dad640 1 --2- 172.21.15.33:0/3905509384 >> v2:172.21.15.138:3300/0 conn(0x7f79600716a0 0x7f79600ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7965dad640 1 -- 172.21.15.33:0/3905509384 >> v1:172.21.15.138:6789/0 conn(0x7f796015e0a0 legacy=0x7f79600b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7965dad640 1 -- 172.21.15.33:0/3905509384 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f796006df00 con 0x7f79600aec20 2024-04-25T01:09:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7965dad640 1 --2- 172.21.15.33:0/3905509384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600aec20 0x7f796006cd50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7960073650 tx=0x7f795801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbb1fc20d582856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7957fff640 1 -- 172.21.15.33:0/3905509384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580a56d0 con 0x7f79600aec20 2024-04-25T01:09:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7957fff640 1 -- 172.21.15.33:0/3905509384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79580a5870 con 0x7f79600aec20 2024-04-25T01:09:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7957fff640 1 -- 172.21.15.33:0/3905509384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79580a5c40 con 0x7f79600aec20 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 -- 172.21.15.33:0/3905509384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600aec20 msgr2=0x7f796006cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 --2- 172.21.15.33:0/3905509384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600aec20 0x7f796006cd50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7960073650 tx=0x7f795801c920 comp rx=0 tx=0).stop 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 -- 172.21.15.33:0/3905509384 shutdown_connections 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 --2- 172.21.15.33:0/3905509384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600aec20 0x7f796006cd50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 --2- 172.21.15.33:0/3905509384 >> v2:172.21.15.138:3300/0 conn(0x7f79600716a0 0x7f79600ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 -- 172.21.15.33:0/3905509384 >> 172.21.15.33:0/3905509384 conn(0x7f79600b1810 msgr2=0x7f79600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 -- 172.21.15.33:0/3905509384 shutdown_connections 2024-04-25T01:09:09.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.140+0000 7f7967810640 1 -- 172.21.15.33:0/3905509384 wait complete. 2024-04-25T01:09:09.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 Processor -- start 2024-04-25T01:09:09.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 -- start start 2024-04-25T01:09:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600716a0 0x7f796013f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79600aec20 0x7f79600ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f796015e0a0 0x7f7960158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79600af0b0 con 0x7f79600716a0 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f796016af70 con 0x7f796015e0a0 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f796015e650 con 0x7f79600aec20 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600716a0 0x7f796013f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600716a0 0x7f796013f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46658/0 (socket says 172.21.15.33:46658) 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 -- 172.21.15.33:0/785510616 learned_addr learned my addr 172.21.15.33:0/785510616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7964dab640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79600aec20 0x7f79600ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7965dad640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f796015e0a0 0x7f7960158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 -- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79600aec20 msgr2=0x7f79600ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79600aec20 0x7f79600ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 -- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f796015e0a0 msgr2=0x7f7960158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f796015e0a0 0x7f7960158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 -- 172.21.15.33:0/785510616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79600cee70 con 0x7f79600716a0 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7965dad640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f796015e0a0 0x7f7960158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f79655ac640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600716a0 0x7f796013f400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f795c00b440 tx=0x7f795c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7955ffb640 1 -- 172.21.15.33:0/785510616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c015020 con 0x7f79600716a0 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7955ffb640 1 -- 172.21.15.33:0/785510616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c011890 con 0x7f79600716a0 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7955ffb640 1 -- 172.21.15.33:0/785510616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c011c80 con 0x7f79600716a0 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79600cf150 con 0x7f79600716a0 2024-04-25T01:09:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f796013f940 con 0x7f79600716a0 2024-04-25T01:09:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7960069040 con 0x7f79600716a0 2024-04-25T01:09:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7955ffb640 1 -- 172.21.15.33:0/785510616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f795c018020 con 0x7f79600716a0 2024-04-25T01:09:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7955ffb640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79440678c0 0x7f7944069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.144+0000 7f7955ffb640 1 -- 172.21.15.33:0/785510616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f795c085df0 con 0x7f79600716a0 2024-04-25T01:09:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.148+0000 7f7964dab640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79440678c0 0x7f7944069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.148+0000 7f7964dab640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79440678c0 0x7f7944069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79600b4870 tx=0x7f7950008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.152+0000 7f7955ffb640 1 -- 172.21.15.33:0/785510616 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f795c01f030 con 0x7f79600716a0 2024-04-25T01:09:09.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.304+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f79600b2d50 con 0x7f79600716a0 2024-04-25T01:09:09.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.304+0000 7f7955ffb640 1 -- 172.21.15.33:0/785510616 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f795c00d040 con 0x7f79600716a0 2024-04-25T01:09:09.298 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:09.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79440678c0 msgr2=0x7f7944069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79440678c0 0x7f7944069d80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f79600b4870 tx=0x7f7950008040 comp rx=0 tx=0).stop 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600716a0 msgr2=0x7f796013f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600716a0 0x7f796013f400 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f795c00b440 tx=0x7f795c00b900 comp rx=0 tx=0).stop 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 shutdown_connections 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79440678c0 0x7f7944069d80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f796015e0a0 0x7f7960158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79600aec20 0x7f79600ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 --2- 172.21.15.33:0/785510616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79600716a0 0x7f796013f400 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 >> 172.21.15.33:0/785510616 conn(0x7f79600b1810 msgr2=0x7f79600af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 shutdown_connections 2024-04-25T01:09:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.308+0000 7f7967810640 1 -- 172.21.15.33:0/785510616 wait complete. 2024-04-25T01:09:09.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd7103c640 1 Processor -- start 2024-04-25T01:09:09.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd7103c640 1 -- start start 2024-04-25T01:09:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd7103c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 0x7fbd6c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd7103c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbd6c0b1980 0x7fbd6c0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd7103c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd6c15c3a0 con 0x7fbd6c15bfc0 2024-04-25T01:09:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd7103c640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbd6c0b14f0 con 0x7fbd6c0b1980 2024-04-25T01:09:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd7103c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6c0cd450 con 0x7fbd6c0d04a0 2024-04-25T01:09:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6ad9d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fbd6c15bfc0 0x7fbd6c0cfd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41080/0 (socket says 172.21.15.33:41080) 2024-04-25T01:09:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6ad9d640 1 -- 172.21.15.33:0/3254064859 learned_addr learned my addr 172.21.15.33:0/3254064859 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6a59c640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 0x7fbd6c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6b59e640 1 --2- 172.21.15.33:0/3254064859 >> v2:172.21.15.138:3300/0 conn(0x7fbd6c0b1980 0x7fbd6c0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2707101817 0 0) 0x7fbd6c15c3a0 con 0x7fbd6c15bfc0 2024-04-25T01:09:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd54003530 con 0x7fbd6c15bfc0 2024-04-25T01:09:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3987951854 0 0) 0x7fbd54003530 con 0x7fbd6c15bfc0 2024-04-25T01:09:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 msgr2=0x7fbd6c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 0x7fbd6c0cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 >> v2:172.21.15.138:3300/0 conn(0x7fbd6c0b1980 msgr2=0x7fbd6c0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:09.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 --2- 172.21.15.33:0/3254064859 >> v2:172.21.15.138:3300/0 conn(0x7fbd6c0b1980 0x7fbd6c0b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c0cd730 con 0x7fbd6c15bfc0 2024-04-25T01:09:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fbd5c001680 con 0x7fbd6c15bfc0 2024-04-25T01:09:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 >> v1:172.21.15.33:6789/0 conn(0x7fbd6c15bfc0 legacy=0x7fbd6c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd540039c0 0x7fbd54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 0x7fbd540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c0b14f0 con 0x7fbd54004810 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c0cd450 con 0x7fbd540039c0 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5400b9d0 con 0x7fbd6c0d04a0 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6a59c640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6ad9d640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd540039c0 0x7fbd54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6b59e640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 0x7fbd540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6a59c640 1 -- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 msgr2=0x7fbd540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6a59c640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd6c0d04a0 0x7fbd540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6a59c640 1 -- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd540039c0 msgr2=0x7fbd54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6a59c640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd540039c0 0x7fbd54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.448+0000 7fbd6a59c640 1 -- 172.21.15.33:0/3254064859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd54003530 con 0x7fbd54004810 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd6a59c640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009ae0 tx=0x7fbd6001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb20b11af21763a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd600096b0 con 0x7fbd54004810 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd600a6480 con 0x7fbd54004810 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd69d9b640 1 -- 172.21.15.33:0/3254064859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd600a6870 con 0x7fbd54004810 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 -- 172.21.15.33:0/3254064859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 msgr2=0x7fbd54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 --2- 172.21.15.33:0/3254064859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd54004810 0x7fbd54004c90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009ae0 tx=0x7fbd6001ca70 comp rx=0 tx=0).stop 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd6b59e640 1 -- 172.21.15.33:0/3254064859 reap_dead start 2024-04-25T01:09:09.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 -- 172.21.15.33:0/3254064859 shutdown_connections 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 -- 172.21.15.33:0/3254064859 >> 172.21.15.33:0/3254064859 conn(0x7fbd6c0d1b00 msgr2=0x7fbd6c0d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 -- 172.21.15.33:0/3254064859 shutdown_connections 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 -- 172.21.15.33:0/3254064859 wait complete. 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 Processor -- start 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.452+0000 7fbd7103c640 1 -- start start 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd54004810 0x7fbd6c06c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c1546c0 0x7fbd6c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c1f4670 0x7fbd6c1f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c0cd230 con 0x7fbd6c1f4670 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c0cd0b0 con 0x7fbd6c1546c0 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6c0d2740 con 0x7fbd54004810 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c1f4670 0x7fbd6c1f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c1f4670 0x7fbd6c1f4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46670/0 (socket says 172.21.15.33:46670) 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 -- 172.21.15.33:0/2493156585 learned_addr learned my addr 172.21.15.33:0/2493156585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:09.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6ad9d640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd54004810 0x7fbd6c06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6a59c640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c1546c0 0x7fbd6c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 -- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd54004810 msgr2=0x7fbd6c06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd54004810 0x7fbd6c06c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 -- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c1546c0 msgr2=0x7fbd6c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c1546c0 0x7fbd6c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 -- 172.21.15.33:0/2493156585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c15c760 con 0x7fbd6c1f4670 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6ad9d640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd54004810 0x7fbd6c06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6b59e640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c1f4670 0x7fbd6c1f4af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009e90 tx=0x7fbd640115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd537fe640 1 -- 172.21.15.33:0/2493156585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64011e80 con 0x7fbd6c1f4670 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd537fe640 1 -- 172.21.15.33:0/2493156585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd6401b070 con 0x7fbd6c1f4670 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c15ca40 con 0x7fbd6c1f4670 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd537fe640 1 -- 172.21.15.33:0/2493156585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640039a0 con 0x7fbd6c1f4670 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c15d000 con 0x7fbd6c1f4670 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6c069040 con 0x7fbd6c1f4670 2024-04-25T01:09:09.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd537fe640 1 -- 172.21.15.33:0/2493156585 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbd64003b40 con 0x7fbd6c1f4670 2024-04-25T01:09:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd537fe640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd580678c0 0x7fbd58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd537fe640 1 -- 172.21.15.33:0/2493156585 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbd64087720 con 0x7fbd6c1f4670 2024-04-25T01:09:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.456+0000 7fbd6ad9d640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd580678c0 0x7fbd58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.460+0000 7fbd6ad9d640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd580678c0 0x7fbd58069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c001060 tx=0x7fbd5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.460+0000 7fbd537fe640 1 -- 172.21.15.33:0/2493156585 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbd64025020 con 0x7fbd6c1f4670 2024-04-25T01:09:09.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbd6c06f260 con 0x7fbd6c1f4670 2024-04-25T01:09:09.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd537fe640 1 -- 172.21.15.33:0/2493156585 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fbd64057f70 con 0x7fbd6c1f4670 2024-04-25T01:09:09.616 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd580678c0 msgr2=0x7fbd58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd580678c0 0x7fbd58069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c001060 tx=0x7fbd5c009040 comp rx=0 tx=0).stop 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c1f4670 msgr2=0x7fbd6c1f4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c1f4670 0x7fbd6c1f4af0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009e90 tx=0x7fbd640115a0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 shutdown_connections 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd580678c0 0x7fbd58069d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd6c1f4670 0x7fbd6c1f4af0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd6c1546c0 0x7fbd6c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 --2- 172.21.15.33:0/2493156585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd54004810 0x7fbd6c06c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 >> 172.21.15.33:0/2493156585 conn(0x7fbd6c0d1b00 msgr2=0x7fbd6c074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 shutdown_connections 2024-04-25T01:09:09.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.624+0000 7fbd7103c640 1 -- 172.21.15.33:0/2493156585 wait complete. 2024-04-25T01:09:09.636 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:09.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.760+0000 7f78fddd6640 1 Processor -- start 2024-04-25T01:09:09.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.760+0000 7f78fddd6640 1 -- start start 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78fddd6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78f8145660 0x7f78f806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78fddd6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f815f960 0x7f78f815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78fddd6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78f8160770 con 0x7f78f81602e0 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78fddd6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f78f806f0a0 con 0x7f78f8145660 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78fddd6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f78f8145a40 con 0x7f78f815f960 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f77fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f78f8145660 0x7f78f806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f7fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f78f81602e0 0x7f78f806d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41086/0 (socket says 172.21.15.33:41086) 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f7fff640 1 -- 172.21.15.33:0/2462288170 learned_addr learned my addr 172.21.15.33:0/2462288170 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78eeffd640 1 --2- 172.21.15.33:0/2462288170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f815f960 0x7f78f815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f77fe640 1 -- 172.21.15.33:0/2462288170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f815f960 msgr2=0x7f78f815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:09.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f77fe640 1 --2- 172.21.15.33:0/2462288170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f815f960 0x7f78f815fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f77fe640 1 -- 172.21.15.33:0/2462288170 >> v1:172.21.15.33:6789/0 conn(0x7f78f81602e0 legacy=0x7f78f806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f6ffd640 1 -- 172.21.15.33:0/2462288170 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3809334391 0 0) 0x7f78f8160770 con 0x7f78f81602e0 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f77fe640 1 -- 172.21.15.33:0/2462288170 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f80cee10 con 0x7f78f8145660 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78eeffd640 1 --2- 172.21.15.33:0/2462288170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f815f960 0x7f78f815fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f77fe640 1 --2- 172.21.15.33:0/2462288170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f806eb60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78e80097c0 tx=0x7f78e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89cfba772279244d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f6ffd640 1 -- 172.21.15.33:0/2462288170 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80b2070 con 0x7f78f8145660 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f6ffd640 1 -- 172.21.15.33:0/2462288170 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e80ad640 con 0x7f78f8145660 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.764+0000 7f78f6ffd640 1 -- 172.21.15.33:0/2462288170 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80ada10 con 0x7f78f8145660 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- 172.21.15.33:0/2462288170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 msgr2=0x7f78f806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 --2- 172.21.15.33:0/2462288170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f806eb60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78e80097c0 tx=0x7f78e801c4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- 172.21.15.33:0/2462288170 shutdown_connections 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 --2- 172.21.15.33:0/2462288170 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f815f960 0x7f78f815fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 --2- 172.21.15.33:0/2462288170 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f806eb60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- 172.21.15.33:0/2462288170 >> 172.21.15.33:0/2462288170 conn(0x7f78f80b1810 msgr2=0x7f78f80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:09.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- 172.21.15.33:0/2462288170 shutdown_connections 2024-04-25T01:09:09.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- 172.21.15.33:0/2462288170 wait complete. 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 Processor -- start 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- start start 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f81573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f815f960 0x7f78f8166580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f81602e0 0x7f78f8143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8168e20 con 0x7f78f8145660 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78f8168ca0 con 0x7f78f815f960 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f78f8168fa0 con 0x7f78f81602e0 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f81573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f81573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46686/0 (socket says 172.21.15.33:46686) 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 -- 172.21.15.33:0/3641678978 learned_addr learned my addr 172.21.15.33:0/3641678978 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 -- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f81602e0 msgr2=0x7f78f8143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f7fff640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f81602e0 0x7f78f8143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78eeffd640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f815f960 0x7f78f8166580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f81602e0 0x7f78f8143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 -- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f815f960 msgr2=0x7f78f8166580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f815f960 0x7f78f8166580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 -- 172.21.15.33:0/3641678978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78f81f8120 con 0x7f78f8145660 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78eeffd640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f815f960 0x7f78f8166580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f7fff640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f81602e0 0x7f78f8143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f77fe640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f81573c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f78e801a0d0 tx=0x7f78e8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f4ff9640 1 -- 172.21.15.33:0/3641678978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80ba070 con 0x7f78f8145660 2024-04-25T01:09:09.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f4ff9640 1 -- 172.21.15.33:0/3641678978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78e80b6410 con 0x7f78f8145660 2024-04-25T01:09:09.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78f81657c0 con 0x7f78f8145660 2024-04-25T01:09:09.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78f4ff9640 1 -- 172.21.15.33:0/3641678978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78e80b2040 con 0x7f78f8145660 2024-04-25T01:09:09.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.768+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78f8165d80 con 0x7f78f8145660 2024-04-25T01:09:09.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.772+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78f8069040 con 0x7f78f8145660 2024-04-25T01:09:09.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.772+0000 7f78f4ff9640 1 -- 172.21.15.33:0/3641678978 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f78e80bd070 con 0x7f78f8145660 2024-04-25T01:09:09.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.772+0000 7f78f4ff9640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78d80678c0 0x7f78d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:09.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.772+0000 7f78f4ff9640 1 -- 172.21.15.33:0/3641678978 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f78e8120020 con 0x7f78f8145660 2024-04-25T01:09:09.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.772+0000 7f78eeffd640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78d80678c0 0x7f78d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:09.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.772+0000 7f78eeffd640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78d80678c0 0x7f78d8069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f78f815f520 tx=0x7f78e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:09.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.776+0000 7f78f4ff9640 1 -- 172.21.15.33:0/3641678978 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78e8016090 con 0x7f78f8145660 2024-04-25T01:09:09.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.936+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f78f8072ac0 con 0x7f78f8145660 2024-04-25T01:09:09.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78f4ff9640 1 -- 172.21.15.33:0/3641678978 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f78e80f0830 con 0x7f78f8145660 2024-04-25T01:09:09.932 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78d80678c0 msgr2=0x7f78d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78d80678c0 0x7f78d8069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f78f815f520 tx=0x7f78e0008040 comp rx=0 tx=0).stop 2024-04-25T01:09:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 msgr2=0x7f78f81573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f81573c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f78e801a0d0 tx=0x7f78e8019240 comp rx=0 tx=0).stop 2024-04-25T01:09:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 shutdown_connections 2024-04-25T01:09:09.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78d80678c0 0x7f78d8069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f78f81602e0 0x7f78f8143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f78f815f960 0x7f78f8166580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 --2- 172.21.15.33:0/3641678978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f78f8145660 0x7f78f81573c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 >> 172.21.15.33:0/3641678978 conn(0x7f78f80b1810 msgr2=0x7f78f80ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 shutdown_connections 2024-04-25T01:09:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:09.940+0000 7f78fddd6640 1 -- 172.21.15.33:0/3641678978 wait complete. 2024-04-25T01:09:09.950 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fba201640 1 Processor -- start 2024-04-25T01:09:10.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fba201640 1 -- start start 2024-04-25T01:09:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fba201640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 0x7f2fb415c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fba201640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb415f9a0 con 0x7f2fb40b5530 2024-04-25T01:09:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fba201640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fb415fc80 con 0x7f2fb40cf0e0 2024-04-25T01:09:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fba201640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb415f750 con 0x7f2fb4145470 2024-04-25T01:09:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb3fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2fb40b5530 0x7f2fb40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41094/0 (socket says 172.21.15.33:41094) 2024-04-25T01:09:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb3fff640 1 -- 172.21.15.33:0/3320885058 learned_addr learned my addr 172.21.15.33:0/3320885058 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:10.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb37fe640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 0x7f2fb415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1711578504 0 0) 0x7f2fb415f9a0 con 0x7f2fb40b5530 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa0003530 con 0x7f2fb40b5530 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1067208486 0 0) 0x7f2fa0003530 con 0x7f2fb40b5530 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 msgr2=0x7f2fb415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 0x7f2fb415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 >> v1:172.21.15.138:6789/0 conn(0x7f2fb40cf0e0 legacy=0x7f2fb40b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb415ff60 con 0x7f2fb40b5530 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f2fac001a00 con 0x7f2fb40b5530 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 >> v1:172.21.15.33:6789/0 conn(0x7f2fb40b5530 legacy=0x7f2fb40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fa00039f0 0x7f2fa0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 0x7f2fa00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb415f750 con 0x7f2fa0004810 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa000ba80 con 0x7f2fa00039f0 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa000bc80 con 0x7f2fb4145470 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fabfff640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb37fe640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fa00039f0 0x7f2fa0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fabfff640 1 -- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 msgr2=0x7f2fa00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb3fff640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 0x7f2fa00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fabfff640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 0x7f2fa00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fabfff640 1 -- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fa00039f0 msgr2=0x7f2fa0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fabfff640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fa00039f0 0x7f2fa0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fabfff640 1 -- 172.21.15.33:0/3320885058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa0003530 con 0x7f2fa0004810 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb37fe640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fa00039f0 0x7f2fa0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:10.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.080+0000 7f2fb3fff640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb4145470 0x7f2fa00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fabfff640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fa0004c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c001750 tx=0x7f2f9c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a333a5969ab58f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0013a0 con 0x7f2fa0004810 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f9c0052f0 con 0x7f2fa0004810 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fb2ffd640 1 -- 172.21.15.33:0/3320885058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c003840 con 0x7f2fa0004810 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- 172.21.15.33:0/3320885058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 msgr2=0x7f2fa0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 --2- 172.21.15.33:0/3320885058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fa0004c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c001750 tx=0x7f2f9c01c3c0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fb3fff640 1 -- 172.21.15.33:0/3320885058 reap_dead start 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- 172.21.15.33:0/3320885058 shutdown_connections 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- 172.21.15.33:0/3320885058 >> 172.21.15.33:0/3320885058 conn(0x7f2fb40b1810 msgr2=0x7f2fb40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- 172.21.15.33:0/3320885058 shutdown_connections 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- 172.21.15.33:0/3320885058 wait complete. 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 Processor -- start 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- start start 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fb40c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb40af440 0x7f2fb415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb40c5970 0x7f2fb40c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb406e850 con 0x7f2fa0004810 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4145850 con 0x7f2fb40c5970 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fba201640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb40cf540 con 0x7f2fb40af440 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fb40c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fb40c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46698/0 (socket says 172.21.15.33:46698) 2024-04-25T01:09:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fb37fe640 1 -- 172.21.15.33:0/2433263159 learned_addr learned my addr 172.21.15.33:0/2433263159 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.084+0000 7f2fabfff640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb40af440 0x7f2fb415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb3fff640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb40c5970 0x7f2fb40c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb37fe640 1 -- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb40af440 msgr2=0x7f2fb415b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb37fe640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb40af440 0x7f2fb415b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb37fe640 1 -- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb40c5970 msgr2=0x7f2fb40c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb37fe640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb40c5970 0x7f2fb40c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb37fe640 1 -- 172.21.15.33:0/2433263159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb40afc90 con 0x7f2fa0004810 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb37fe640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fb40c4e40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40116d0 tx=0x7f2fa4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb0ff9640 1 -- 172.21.15.33:0/2433263159 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400f9c0 con 0x7f2fa0004810 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb0ff9640 1 -- 172.21.15.33:0/2433263159 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa400fb60 con 0x7f2fa0004810 2024-04-25T01:09:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb40aff70 con 0x7f2fa0004810 2024-04-25T01:09:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb0ff9640 1 -- 172.21.15.33:0/2433263159 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4015460 con 0x7f2fa0004810 2024-04-25T01:09:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb40ac490 con 0x7f2fa0004810 2024-04-25T01:09:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4069040 con 0x7f2fa0004810 2024-04-25T01:09:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb0ff9640 1 -- 172.21.15.33:0/2433263159 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2fa4015600 con 0x7f2fa0004810 2024-04-25T01:09:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb0ff9640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f940678c0 0x7f2f94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fb0ff9640 1 -- 172.21.15.33:0/2433263159 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2fa4086b40 con 0x7f2fa0004810 2024-04-25T01:09:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fabfff640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f940678c0 0x7f2f94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.088+0000 7f2fabfff640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f940678c0 0x7f2f94069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00057f0 tx=0x7f2f9c0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.092+0000 7f2fb0ff9640 1 -- 172.21.15.33:0/2433263159 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2fa4019030 con 0x7f2fa0004810 2024-04-25T01:09:10.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.256+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fb4071420 con 0x7f2fa0004810 2024-04-25T01:09:10.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fb0ff9640 1 -- 172.21.15.33:0/2433263159 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18502 (secure 0 0 0) 0x7f2fa40572c0 con 0x7f2fa0004810 2024-04-25T01:09:10.252 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:10.252 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:08:28.906693+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f940678c0 msgr2=0x7f2f94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f940678c0 0x7f2f94069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2fa00057f0 tx=0x7f2f9c0011a0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 msgr2=0x7f2fb40c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fb40c4e40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40116d0 tx=0x7f2fa4011b90 comp rx=0 tx=0).stop 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 shutdown_connections 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f940678c0 0x7f2f94069d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fb40c5970 0x7f2fb40c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fb40af440 0x7f2fb415b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 --2- 172.21.15.33:0/2433263159 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fa0004810 0x7f2fb40c4e40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 >> 172.21.15.33:0/2433263159 conn(0x7f2fb40b1810 msgr2=0x7f2fb40b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:10.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 shutdown_connections 2024-04-25T01:09:10.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.260+0000 7f2fba201640 1 -- 172.21.15.33:0/2433263159 wait complete. 2024-04-25T01:09:10.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:10.272 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:10.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a3c74640 1 Processor -- start 2024-04-25T01:09:10.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a3c74640 1 -- start start 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a3c74640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb79c0d1f60 0x7fb79c0d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a3c74640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb79c0b9a20 0x7fb79c0d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a3c74640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb79c0b6cc0 con 0x7fb79c15e0d0 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a3c74640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb79c0cfb00 con 0x7fb79c0d1f60 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a3c74640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb79c0b7580 con 0x7fb79c0b9a20 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a120f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb79c0d1f60 0x7fb79c0d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a1a10640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb79c15e0d0 0x7fb79c0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41104/0 (socket says 172.21.15.33:41104) 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a1a10640 1 -- 172.21.15.33:0/3335106373 learned_addr learned my addr 172.21.15.33:0/3335106373 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a120f640 1 -- 172.21.15.33:0/3335106373 >> v2:172.21.15.138:3300/0 conn(0x7fb79c0b9a20 msgr2=0x7fb79c0d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a120f640 1 --2- 172.21.15.33:0/3335106373 >> v2:172.21.15.138:3300/0 conn(0x7fb79c0b9a20 0x7fb79c0d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a120f640 1 -- 172.21.15.33:0/3335106373 >> v1:172.21.15.33:6789/0 conn(0x7fb79c15e0d0 legacy=0x7fb79c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.404+0000 7fb7a120f640 1 -- 172.21.15.33:0/3335106373 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c0b6f30 con 0x7fb79c0d1f60 2024-04-25T01:09:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 --2- 172.21.15.33:0/3335106373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c0d23b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb79c08c7c0 tx=0x7fb79801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf90c6d4159b5aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a0a0e640 1 -- 172.21.15.33:0/3335106373 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7980a7070 con 0x7fb79c0d1f60 2024-04-25T01:09:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a0a0e640 1 -- 172.21.15.33:0/3335106373 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7980b0070 con 0x7fb79c0d1f60 2024-04-25T01:09:10.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a0a0e640 1 -- 172.21.15.33:0/3335106373 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7980ab890 con 0x7fb79c0d1f60 2024-04-25T01:09:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- 172.21.15.33:0/3335106373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 msgr2=0x7fb79c0d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/3335106373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c0d23b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb79c08c7c0 tx=0x7fb79801c450 comp rx=0 tx=0).stop 2024-04-25T01:09:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- 172.21.15.33:0/3335106373 shutdown_connections 2024-04-25T01:09:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/3335106373 >> v2:172.21.15.138:3300/0 conn(0x7fb79c0b9a20 0x7fb79c0d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/3335106373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c0d23b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- 172.21.15.33:0/3335106373 >> 172.21.15.33:0/3335106373 conn(0x7fb79c0d3dd0 msgr2=0x7fb79c0d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:10.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- 172.21.15.33:0/3335106373 shutdown_connections 2024-04-25T01:09:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- 172.21.15.33:0/3335106373 wait complete. 2024-04-25T01:09:10.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 Processor -- start 2024-04-25T01:09:10.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- start start 2024-04-25T01:09:10.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79c0b9a20 0x7fb79c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb79c15e0d0 0x7fb79c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c16a690 con 0x7fb79c0d1f60 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c16a510 con 0x7fb79c0b9a20 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a3c74640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb79c16a810 con 0x7fb79c15e0d0 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46716/0 (socket says 172.21.15.33:46716) 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 -- 172.21.15.33:0/2606433376 learned_addr learned my addr 172.21.15.33:0/2606433376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a1a10640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79c0b9a20 0x7fb79c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 -- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb79c15e0d0 msgr2=0x7fb79c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a2211640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb79c15e0d0 0x7fb79c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb79c15e0d0 0x7fb79c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 -- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79c0b9a20 msgr2=0x7fb79c159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79c0b9a20 0x7fb79c159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 -- 172.21.15.33:0/2606433376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c15a970 con 0x7fb79c0d1f60 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a1a10640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79c0b9a20 0x7fb79c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a2211640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb79c15e0d0 0x7fb79c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.408+0000 7fb7a120f640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c15a0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb798019060 tx=0x7fb79801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb78e7fc640 1 -- 172.21.15.33:0/2606433376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7980bf610 con 0x7fb79c0d1f60 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb78e7fc640 1 -- 172.21.15.33:0/2606433376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7980bf7b0 con 0x7fb79c0d1f60 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb78e7fc640 1 -- 172.21.15.33:0/2606433376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7980bfb60 con 0x7fb79c0d1f60 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c15ac50 con 0x7fb79c0d1f60 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c15b210 con 0x7fb79c0d1f60 2024-04-25T01:09:10.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c069040 con 0x7fb79c0d1f60 2024-04-25T01:09:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb78e7fc640 1 -- 172.21.15.33:0/2606433376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7980bd020 con 0x7fb79c0d1f60 2024-04-25T01:09:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb78e7fc640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7800678c0 0x7fb780069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb78e7fc640 1 -- 172.21.15.33:0/2606433376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb798120c50 con 0x7fb79c0d1f60 2024-04-25T01:09:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb7a1a10640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7800678c0 0x7fb780069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.412+0000 7fb7a1a10640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7800678c0 0x7fb780069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb79c06c940 tx=0x7fb790009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.416+0000 7fb78e7fc640 1 -- 172.21.15.33:0/2606433376 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb798016070 con 0x7fb79c0d1f60 2024-04-25T01:09:10.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb79c161710 con 0x7fb79c0d1f60 2024-04-25T01:09:10.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb78e7fc640 1 -- 172.21.15.33:0/2606433376 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fb7980f13b0 con 0x7fb79c0d1f60 2024-04-25T01:09:10.572 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7800678c0 msgr2=0x7fb780069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7800678c0 0x7fb780069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb79c06c940 tx=0x7fb790009040 comp rx=0 tx=0).stop 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 msgr2=0x7fb79c15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c15a0b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb798019060 tx=0x7fb79801ac90 comp rx=0 tx=0).stop 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 shutdown_connections 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7800678c0 0x7fb780069d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb79c15e0d0 0x7fb79c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb79c0d1f60 0x7fb79c15a0b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 --2- 172.21.15.33:0/2606433376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79c0b9a20 0x7fb79c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 >> 172.21.15.33:0/2606433376 conn(0x7fb79c0d3dd0 msgr2=0x7fb79c0b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 shutdown_connections 2024-04-25T01:09:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.580+0000 7fb7a3c74640 1 -- 172.21.15.33:0/2606433376 wait complete. 2024-04-25T01:09:10.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:10.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f027ea640 1 Processor -- start 2024-04-25T01:09:10.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f027ea640 1 -- start start 2024-04-25T01:09:10.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f027ea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc0d23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f027ea640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0efc0cfa20 0x7f0efc161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f027ea640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0efc0d29f0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f027ea640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0efc0ceae0 con 0x7f0efc0cfa20 2024-04-25T01:09:10.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f027ea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc0cedc0 con 0x7f0efc161720 2024-04-25T01:09:10.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f00d87640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0efc0cfa20 0x7f0efc161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f00d87640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0efc0cfa20 0x7f0efc161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46718/0 (socket says 172.21.15.33:46718) 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f00d87640 1 -- 172.21.15.33:0/4227057045 learned_addr learned my addr 172.21.15.33:0/4227057045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0efb7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0efc0cf5c0 0x7f0efc0b3660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41116/0 (socket says 172.21.15.33:41116) 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0efbfff640 1 --2- 172.21.15.33:0/4227057045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc0d23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f00d87640 1 -- 172.21.15.33:0/4227057045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 msgr2=0x7f0efc0d23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f00d87640 1 --2- 172.21.15.33:0/4227057045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc0d23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f00d87640 1 -- 172.21.15.33:0/4227057045 >> v1:172.21.15.33:6789/0 conn(0x7f0efc0cf5c0 legacy=0x7f0efc0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0f00d87640 1 -- 172.21.15.33:0/4227057045 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc0cf0a0 con 0x7f0efc0cfa20 2024-04-25T01:09:10.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.728+0000 7f0efaffd640 1 -- 172.21.15.33:0/4227057045 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2202775804 0 0) 0x7f0efc0d29f0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f00d87640 1 --2- 172.21.15.33:0/4227057045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cfa20 0x7f0efc161c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009870 tx=0x7f0ef401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a38f711bae5fdc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efaffd640 1 -- 172.21.15.33:0/4227057045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40b2070 con 0x7f0efc0cfa20 2024-04-25T01:09:10.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efaffd640 1 -- 172.21.15.33:0/4227057045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ef40a9070 con 0x7f0efc0cfa20 2024-04-25T01:09:10.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efaffd640 1 -- 172.21.15.33:0/4227057045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ef40ad880 con 0x7f0efc0cfa20 2024-04-25T01:09:10.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- 172.21.15.33:0/4227057045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cfa20 msgr2=0x7f0efc161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 --2- 172.21.15.33:0/4227057045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cfa20 0x7f0efc161c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4009870 tx=0x7f0ef401c560 comp rx=0 tx=0).stop 2024-04-25T01:09:10.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- 172.21.15.33:0/4227057045 shutdown_connections 2024-04-25T01:09:10.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 --2- 172.21.15.33:0/4227057045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cfa20 0x7f0efc161c10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 --2- 172.21.15.33:0/4227057045 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc0d23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- 172.21.15.33:0/4227057045 >> 172.21.15.33:0/4227057045 conn(0x7f0efc0d3e30 msgr2=0x7f0efc0d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:10.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- 172.21.15.33:0/4227057045 shutdown_connections 2024-04-25T01:09:10.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- 172.21.15.33:0/4227057045 wait complete. 2024-04-25T01:09:10.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 Processor -- start 2024-04-25T01:09:10.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- start start 2024-04-25T01:09:10.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc15c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc0cfa20 0x7f0efc150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc16b0d0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc16af50 con 0x7f0efc0cfa20 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f027ea640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc16b250 con 0x7f0efc161720 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc15c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc15c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46724/0 (socket says 172.21.15.33:46724) 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 -- 172.21.15.33:0/862405336 learned_addr learned my addr 172.21.15.33:0/862405336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0f00d87640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efb7fe640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc0cfa20 0x7f0efc150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 -- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 msgr2=0x7f0efc15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 -- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc0cfa20 msgr2=0x7f0efc150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc0cfa20 0x7f0efc150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efbfff640 1 -- 172.21.15.33:0/862405336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc15fce0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.732+0000 7f0efb7fe640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc0cfa20 0x7f0efc150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:10.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0efbfff640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc15c640 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0eec007e40 tx=0x7f0eec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0ef8ff9640 1 -- 172.21.15.33:0/862405336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec014620 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0ef8ff9640 1 -- 172.21.15.33:0/862405336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec019070 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc0b7ac0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0ef8ff9640 1 -- 172.21.15.33:0/862405336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec014a00 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc15cfb0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0efc069040 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0ef8ff9640 1 -- 172.21.15.33:0/862405336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0eec014ba0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0ef8ff9640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0ee80678c0 0x7f0ee8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0ef8ff9640 1 -- 172.21.15.33:0/862405336 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0eec086890 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0efb7fe640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0ee80678c0 0x7f0ee8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:10.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.736+0000 7f0efb7fe640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0ee80678c0 0x7f0ee8069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0efc161140 tx=0x7f0ef0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:10.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.740+0000 7f0ef8ff9640 1 -- 172.21.15.33:0/862405336 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0efc15fce0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.904+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0efc0b9cf0 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.904+0000 7f0ef8ff9640 1 -- 172.21.15.33:0/862405336 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f0eec057090 con 0x7f0efc0cf5c0 2024-04-25T01:09:10.898 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0ee80678c0 msgr2=0x7f0ee8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0ee80678c0 0x7f0ee8069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0efc161140 tx=0x7f0ef0009040 comp rx=0 tx=0).stop 2024-04-25T01:09:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cf5c0 msgr2=0x7f0efc15c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc15c640 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0eec007e40 tx=0x7f0eec00e4b0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 shutdown_connections 2024-04-25T01:09:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0ee80678c0 0x7f0ee8069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0efc161720 0x7f0efc15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0efc0cfa20 0x7f0efc150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 --2- 172.21.15.33:0/862405336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0efc0cf5c0 0x7f0efc15c640 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 >> 172.21.15.33:0/862405336 conn(0x7f0efc0d3e30 msgr2=0x7f0efc0d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 shutdown_connections 2024-04-25T01:09:10.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:10.908+0000 7f0f027ea640 1 -- 172.21.15.33:0/862405336 wait complete. 2024-04-25T01:09:10.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:11.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.040+0000 7f23e74f7640 1 Processor -- start 2024-04-25T01:09:11.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e74f7640 1 -- start start 2024-04-25T01:09:11.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e74f7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e006dfe0 0x7f23e00b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e74f7640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f23e00ce3f0 0x7f23e00b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e74f7640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23e006ff60 con 0x7f23e00cdf90 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e74f7640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f23e00b3f60 con 0x7f23e00ce3f0 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e74f7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f23e00b4190 con 0x7f23e006dfe0 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e4a92640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f23e00cdf90 0x7f23e00b5230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41124/0 (socket says 172.21.15.33:41124) 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5a94640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f23e00ce3f0 0x7f23e00b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e4a92640 1 -- 172.21.15.33:0/414108193 learned_addr learned my addr 172.21.15.33:0/414108193 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5293640 1 --2- 172.21.15.33:0/414108193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e006dfe0 0x7f23e00b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5a94640 1 -- 172.21.15.33:0/414108193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e006dfe0 msgr2=0x7f23e00b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5a94640 1 --2- 172.21.15.33:0/414108193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e006dfe0 0x7f23e00b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5a94640 1 -- 172.21.15.33:0/414108193 >> v1:172.21.15.33:6789/0 conn(0x7f23e00cdf90 legacy=0x7f23e00b5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5a94640 1 -- 172.21.15.33:0/414108193 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e00b4470 con 0x7f23e00ce3f0 2024-04-25T01:09:11.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5293640 1 --2- 172.21.15.33:0/414108193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e006dfe0 0x7f23e00b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:11.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23e5a94640 1 --2- 172.21.15.33:0/414108193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00ce3f0 0x7f23e00b5940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009d90 tx=0x7f23d801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69f816f695a413f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23d7fff640 1 -- 172.21.15.33:0/414108193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d80b2070 con 0x7f23e00ce3f0 2024-04-25T01:09:11.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23d7fff640 1 -- 172.21.15.33:0/414108193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23d80a5a00 con 0x7f23e00ce3f0 2024-04-25T01:09:11.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.044+0000 7f23d7fff640 1 -- 172.21.15.33:0/414108193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d80a5dd0 con 0x7f23e00ce3f0 2024-04-25T01:09:11.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- 172.21.15.33:0/414108193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00ce3f0 msgr2=0x7f23e00b5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 --2- 172.21.15.33:0/414108193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00ce3f0 0x7f23e00b5940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f23d8009d90 tx=0x7f23d801c940 comp rx=0 tx=0).stop 2024-04-25T01:09:11.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- 172.21.15.33:0/414108193 shutdown_connections 2024-04-25T01:09:11.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 --2- 172.21.15.33:0/414108193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00ce3f0 0x7f23e00b5940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 --2- 172.21.15.33:0/414108193 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e006dfe0 0x7f23e00b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- 172.21.15.33:0/414108193 >> 172.21.15.33:0/414108193 conn(0x7f23e00d0a90 msgr2=0x7f23e00d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:11.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- 172.21.15.33:0/414108193 shutdown_connections 2024-04-25T01:09:11.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- 172.21.15.33:0/414108193 wait complete. 2024-04-25T01:09:11.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 Processor -- start 2024-04-25T01:09:11.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- start start 2024-04-25T01:09:11.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e006dfe0 0x7f23e014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00cdf90 0x7f23e013d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e00ce3f0 0x7f23e013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23e006ed50 con 0x7f23e00cdf90 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f23e006ebd0 con 0x7f23e006dfe0 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f23e006eed0 con 0x7f23e00ce3f0 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00cdf90 0x7f23e013d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00cdf90 0x7f23e013d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46742/0 (socket says 172.21.15.33:46742) 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 -- 172.21.15.33:0/1755227288 learned_addr learned my addr 172.21.15.33:0/1755227288 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e5a94640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e00ce3f0 0x7f23e013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e5293640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e006dfe0 0x7f23e014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 -- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e00ce3f0 msgr2=0x7f23e013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e00ce3f0 0x7f23e013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 -- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e006dfe0 msgr2=0x7f23e014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e006dfe0 0x7f23e014e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 -- 172.21.15.33:0/1755227288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e014ca40 con 0x7f23e00cdf90 2024-04-25T01:09:11.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e5a94640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e00ce3f0 0x7f23e013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:11.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e4a92640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00cdf90 0x7f23e013d4c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c370 tx=0x7f23d000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23d5ffb640 1 -- 172.21.15.33:0/1755227288 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0017070 con 0x7f23e00cdf90 2024-04-25T01:09:11.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23d5ffb640 1 -- 172.21.15.33:0/1755227288 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23d0005cf0 con 0x7f23e00cdf90 2024-04-25T01:09:11.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23d5ffb640 1 -- 172.21.15.33:0/1755227288 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d00072f0 con 0x7f23e00cdf90 2024-04-25T01:09:11.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e014c010 con 0x7f23e00cdf90 2024-04-25T01:09:11.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.048+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e014cce0 con 0x7f23e00cdf90 2024-04-25T01:09:11.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.052+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23e0069040 con 0x7f23e00cdf90 2024-04-25T01:09:11.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.052+0000 7f23d5ffb640 1 -- 172.21.15.33:0/1755227288 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23d001a070 con 0x7f23e00cdf90 2024-04-25T01:09:11.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.052+0000 7f23d5ffb640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f23c40678f0 0x7f23c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.052+0000 7f23d5ffb640 1 -- 172.21.15.33:0/1755227288 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f23d0085250 con 0x7f23e00cdf90 2024-04-25T01:09:11.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.052+0000 7f23e5293640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f23c40678f0 0x7f23c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.052+0000 7f23e5293640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f23c40678f0 0x7f23c4069db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23dc006fd0 tx=0x7f23dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.056+0000 7f23d5ffb640 1 -- 172.21.15.33:0/1755227288 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f23d0055ae0 con 0x7f23e00cdf90 2024-04-25T01:09:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.212+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f23e0155cf0 con 0x7f23e00cdf90 2024-04-25T01:09:11.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.212+0000 7f23d5ffb640 1 -- 172.21.15.33:0/1755227288 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f23d00553e0 con 0x7f23e00cdf90 2024-04-25T01:09:11.206 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f23c40678f0 msgr2=0x7f23c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f23c40678f0 0x7f23c4069db0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f23dc006fd0 tx=0x7f23dc008040 comp rx=0 tx=0).stop 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00cdf90 msgr2=0x7f23e013d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00cdf90 0x7f23e013d4c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c370 tx=0x7f23d000c830 comp rx=0 tx=0).stop 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 shutdown_connections 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f23c40678f0 0x7f23c4069db0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f23e00ce3f0 0x7f23e013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23e00cdf90 0x7f23e013d4c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 --2- 172.21.15.33:0/1755227288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23e006dfe0 0x7f23e014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 >> 172.21.15.33:0/1755227288 conn(0x7f23e00d0a90 msgr2=0x7f23e006b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:11.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 shutdown_connections 2024-04-25T01:09:11.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.216+0000 7f23e74f7640 1 -- 172.21.15.33:0/1755227288 wait complete. 2024-04-25T01:09:11.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:11.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.348+0000 7f789a50b640 1 Processor -- start 2024-04-25T01:09:11.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.348+0000 7f789a50b640 1 -- start start 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f789a50b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f789415ae20 0x7f78940b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f789a50b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f789406ecc0 0x7f78940b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f789a50b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78940b4a20 con 0x7f7894072410 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f789a50b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f78940b2bb0 con 0x7f789415ae20 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f789a50b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f78940b2e90 con 0x7f789406ecc0 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f789415ae20 0x7f78940b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f789415ae20 0x7f78940b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46756/0 (socket says 172.21.15.33:46756) 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 -- 172.21.15.33:0/1878928980 learned_addr learned my addr 172.21.15.33:0/1878928980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:11.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 -- 172.21.15.33:0/1878928980 >> v2:172.21.15.138:3300/0 conn(0x7f789406ecc0 msgr2=0x7f78940b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 --2- 172.21.15.33:0/1878928980 >> v2:172.21.15.138:3300/0 conn(0x7f789406ecc0 0x7f78940b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 -- 172.21.15.33:0/1878928980 >> v1:172.21.15.138:6789/0 conn(0x7f7894072410 legacy=0x7f789406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:11.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 -- 172.21.15.33:0/1878928980 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78940b3170 con 0x7f789415ae20 2024-04-25T01:09:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7893fff640 1 --2- 172.21.15.33:0/1878928980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789415ae20 0x7f78940b3690 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f78940cfa10 tx=0x7f788401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffce9194a6528fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7892ffd640 1 -- 172.21.15.33:0/1878928980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840b4070 con 0x7f789415ae20 2024-04-25T01:09:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7892ffd640 1 -- 172.21.15.33:0/1878928980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78840a7cc0 con 0x7f789415ae20 2024-04-25T01:09:11.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.352+0000 7f7892ffd640 1 -- 172.21.15.33:0/1878928980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840af660 con 0x7f789415ae20 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- 172.21.15.33:0/1878928980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789415ae20 msgr2=0x7f78940b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 --2- 172.21.15.33:0/1878928980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789415ae20 0x7f78940b3690 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f78940cfa10 tx=0x7f788401ebd0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- 172.21.15.33:0/1878928980 shutdown_connections 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 --2- 172.21.15.33:0/1878928980 >> v2:172.21.15.138:3300/0 conn(0x7f789406ecc0 0x7f78940b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 --2- 172.21.15.33:0/1878928980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789415ae20 0x7f78940b3690 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- 172.21.15.33:0/1878928980 >> 172.21.15.33:0/1878928980 conn(0x7f78940b1810 msgr2=0x7f78940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- 172.21.15.33:0/1878928980 shutdown_connections 2024-04-25T01:09:11.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- 172.21.15.33:0/1878928980 wait complete. 2024-04-25T01:09:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 Processor -- start 2024-04-25T01:09:11.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- start start 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789406ecc0 0x7f78941f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7894072410 0x7f789414dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f789415ae20 0x7f789415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f78940cda50 con 0x7f789406ecc0 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78940cd8d0 con 0x7f7894072410 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f78940cdbd0 con 0x7f789415ae20 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789406ecc0 0x7f78941f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789406ecc0 0x7f78941f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46766/0 (socket says 172.21.15.33:46766) 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 -- 172.21.15.33:0/2105404384 learned_addr learned my addr 172.21.15.33:0/2105404384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7898aa8640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f789415ae20 0x7f789415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 -- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f789415ae20 msgr2=0x7f789415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f789415ae20 0x7f789415d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 -- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7894072410 msgr2=0x7f789414dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f78937fe640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7894072410 0x7f789414dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7894072410 0x7f789414dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 -- 172.21.15.33:0/2105404384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78941f2e40 con 0x7f789406ecc0 2024-04-25T01:09:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7898aa8640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f789415ae20 0x7f789415d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7893fff640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789406ecc0 0x7f78941f3be0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f788400b2c0 tx=0x7f788401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7890ff9640 1 -- 172.21.15.33:0/2105404384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840be070 con 0x7f789406ecc0 2024-04-25T01:09:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789414e3e0 con 0x7f789406ecc0 2024-04-25T01:09:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7890ff9640 1 -- 172.21.15.33:0/2105404384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78840afc10 con 0x7f789406ecc0 2024-04-25T01:09:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f7890ff9640 1 -- 172.21.15.33:0/2105404384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78840b4040 con 0x7f789406ecc0 2024-04-25T01:09:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.356+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78941f33e0 con 0x7f789406ecc0 2024-04-25T01:09:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.360+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7894069040 con 0x7f789406ecc0 2024-04-25T01:09:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.360+0000 7f7890ff9640 1 -- 172.21.15.33:0/2105404384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f78840c1070 con 0x7f789406ecc0 2024-04-25T01:09:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.360+0000 7f7890ff9640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78800678f0 0x7f7880069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.360+0000 7f7890ff9640 1 -- 172.21.15.33:0/2105404384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f7884123330 con 0x7f789406ecc0 2024-04-25T01:09:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.360+0000 7f78937fe640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78800678f0 0x7f7880069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.360+0000 7f78937fe640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78800678f0 0x7f7880069db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78941483b0 tx=0x7f7888009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.364+0000 7f7890ff9640 1 -- 172.21.15.33:0/2105404384 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f78840187c0 con 0x7f789406ecc0 2024-04-25T01:09:11.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.524+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f789406ce50 con 0x7f789406ecc0 2024-04-25T01:09:11.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.524+0000 7f7890ff9640 1 -- 172.21.15.33:0/2105404384 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) ==== 103+0+11 (secure 0 0 0) 0x7f78840f39e0 con 0x7f789406ecc0 2024-04-25T01:09:11.518 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78800678f0 msgr2=0x7f7880069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78800678f0 0x7f7880069db0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f78941483b0 tx=0x7f7888009040 comp rx=0 tx=0).stop 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789406ecc0 msgr2=0x7f78941f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789406ecc0 0x7f78941f3be0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f788400b2c0 tx=0x7f788401b970 comp rx=0 tx=0).stop 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 shutdown_connections 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f78800678f0 0x7f7880069db0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f789415ae20 0x7f789415d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7894072410 0x7f789414dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 --2- 172.21.15.33:0/2105404384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f789406ecc0 0x7f78941f3be0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 >> 172.21.15.33:0/2105404384 conn(0x7f78940b1810 msgr2=0x7f789406e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:11.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 shutdown_connections 2024-04-25T01:09:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.528+0000 7f789a50b640 1 -- 172.21.15.33:0/2105404384 wait complete. 2024-04-25T01:09:11.591 INFO:tasks.cephfs_test_runner:Starting test: test_acls (tasks.cephfs.test_acls.TestACLs) 2024-04-25T01:09:11.592 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:11.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f480df17640 1 Processor -- start 2024-04-25T01:09:11.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f480df17640 1 -- start start 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f480df17640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 0x7f48080b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f480df17640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48081448d0 con 0x7f4808072540 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f480df17640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48080b5b60 con 0x7f480815e660 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f480df17640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f48080ce0c0 con 0x7f48081444f0 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f4807fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4808072540 0x7f48080ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41132/0 (socket says 172.21.15.33:41132) 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f4807fff640 1 -- 172.21.15.33:0/3550540729 learned_addr learned my addr 172.21.15.33:0/3550540729 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f48077fe640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 0x7f48080b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2532292373 0 0) 0x7f48000014a0 con 0x7f4808072540 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.724+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47f4003530 con 0x7f4808072540 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4248789012 0 0) 0x7f47f4003530 con 0x7f4808072540 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 msgr2=0x7f48080b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 0x7f48080b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 >> v1:172.21.15.138:6789/0 conn(0x7f480815e660 legacy=0x7f4808071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480815ec20 con 0x7f4808072540 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f4800001bd0 con 0x7f4808072540 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 >> v1:172.21.15.33:6789/0 conn(0x7f4808072540 legacy=0x7f48080ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 0x7f47f40039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f4003f70 0x7f47f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f4004d50 0x7f47f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48080ce0c0 con 0x7f47f4003f70 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47f400bb60 con 0x7f47f4004d50 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47f400bd60 con 0x7f48081444f0 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f47feffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f4003f70 0x7f47f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f48077fe640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 0x7f47f40039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4807fff640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f4004d50 0x7f47f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f47feffd640 1 -- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 msgr2=0x7f47f40039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f47feffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 0x7f47f40039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f47feffd640 1 -- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f4004d50 msgr2=0x7f47f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f47feffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f4004d50 0x7f47f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f47feffd640 1 -- 172.21.15.33:0/3550540729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f4003530 con 0x7f47f4003f70 2024-04-25T01:09:11.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f48077fe640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f48081444f0 0x7f47f40039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f47feffd640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f4003f70 0x7f47f4004810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f47f0001780 tx=0x7f47f001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38410ef78d77b70b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f00013a0 con 0x7f47f4003f70 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f00052f0 con 0x7f47f4003f70 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4806ffd640 1 -- 172.21.15.33:0/3550540729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0003840 con 0x7f47f4003f70 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f480df17640 1 -- 172.21.15.33:0/3550540729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f4003f70 msgr2=0x7f47f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f480df17640 1 --2- 172.21.15.33:0/3550540729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f4003f70 0x7f47f4004810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f47f0001780 tx=0x7f47f001c420 comp rx=0 tx=0).stop 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f4807fff640 1 -- 172.21.15.33:0/3550540729 reap_dead start 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f480df17640 1 -- 172.21.15.33:0/3550540729 shutdown_connections 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f480df17640 1 -- 172.21.15.33:0/3550540729 >> 172.21.15.33:0/3550540729 conn(0x7f48080b1810 msgr2=0x7f48080b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f480df17640 1 -- 172.21.15.33:0/3550540729 shutdown_connections 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.728+0000 7f480df17640 1 -- 172.21.15.33:0/3550540729 wait complete. 2024-04-25T01:09:11.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 Processor -- start 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 -- start start 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f4003f70 0x7f48081f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4808135350 0x7f480813a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f480813ad40 0x7f4808137840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f48080b37a0 con 0x7f480813ad40 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48080b3620 con 0x7f4808135350 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f4808166a00 con 0x7f47f4003f70 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f480813ad40 0x7f4808137840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f480813ad40 0x7f4808137840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46786/0 (socket says 172.21.15.33:46786) 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 -- 172.21.15.33:0/2129063642 learned_addr learned my addr 172.21.15.33:0/2129063642 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f47feffd640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4808135350 0x7f480813a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f48077fe640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f4003f70 0x7f48081f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 -- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f4003f70 msgr2=0x7f48081f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f4003f70 0x7f48081f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 -- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4808135350 msgr2=0x7f480813a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4808135350 0x7f480813a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 -- 172.21.15.33:0/2129063642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480815b500 con 0x7f480813ad40 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f47feffd640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4808135350 0x7f480813a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4807fff640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f480813ad40 0x7f4808137840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48080cd4b0 tx=0x7f4800007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4804ff9640 1 -- 172.21.15.33:0/2129063642 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480000ea20 con 0x7f480813ad40 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4804ff9640 1 -- 172.21.15.33:0/2129063642 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f480000ebe0 con 0x7f480813ad40 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4804ff9640 1 -- 172.21.15.33:0/2129063642 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480000edd0 con 0x7f480813ad40 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f480815b7c0 con 0x7f480813ad40 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f480815bd80 con 0x7f480813ad40 2024-04-25T01:09:11.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4808069040 con 0x7f480813ad40 2024-04-25T01:09:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.732+0000 7f4804ff9640 1 -- 172.21.15.33:0/2129063642 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f480001a020 con 0x7f480813ad40 2024-04-25T01:09:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.736+0000 7f4804ff9640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47e80678f0 0x7f47e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.736+0000 7f4804ff9640 1 -- 172.21.15.33:0/2129063642 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f48000858d0 con 0x7f480813ad40 2024-04-25T01:09:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.736+0000 7f48077fe640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47e80678f0 0x7f47e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.736+0000 7f48077fe640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47e80678f0 0x7f47e8069db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ab0 tx=0x7f47f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:11.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.740+0000 7f4804ff9640 1 -- 172.21.15.33:0/2129063642 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4800021020 con 0x7f480813ad40 2024-04-25T01:09:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.896+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f480806e8e0 con 0x7f480813ad40 2024-04-25T01:09:11.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.900+0000 7f4804ff9640 1 -- 172.21.15.33:0/2129063642 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18890 (secure 0 0 0) 0x7f4800083e20 con 0x7f480813ad40 2024-04-25T01:09:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:11.894 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":26,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:11.307778+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:11.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47e80678f0 msgr2=0x7f47e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47e80678f0 0x7f47e8069db0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f47f8009ab0 tx=0x7f47f8008040 comp rx=0 tx=0).stop 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f480813ad40 msgr2=0x7f4808137840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f480813ad40 0x7f4808137840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f48080cd4b0 tx=0x7f4800007ba0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 shutdown_connections 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47e80678f0 0x7f47e8069db0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f480813ad40 0x7f4808137840 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4808135350 0x7f480813a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 --2- 172.21.15.33:0/2129063642 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f4003f70 0x7f48081f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 >> 172.21.15.33:0/2129063642 conn(0x7f48080b1810 msgr2=0x7f480806e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 shutdown_connections 2024-04-25T01:09:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:11.904+0000 7f480df17640 1 -- 172.21.15.33:0/2129063642 wait complete. 2024-04-25T01:09:11.912 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:11.913 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:12.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b15419640 1 Processor -- start 2024-04-25T01:09:12.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b15419640 1 -- start start 2024-04-25T01:09:12.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b15419640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 0x7f9b1006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b15419640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9b101604b0 0x7f9b10071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b15419640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b100cee10 con 0x7f9b1015f960 2024-04-25T01:09:12.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b15419640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1006f0a0 con 0x7f9b101604b0 2024-04-25T01:09:12.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b15419640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b10145a40 con 0x7f9b10145660 2024-04-25T01:09:12.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0e7fc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f9b1015f960 0x7f9b1015fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41138/0 (socket says 172.21.15.33:41138) 2024-04-25T01:09:12.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0e7fc640 1 -- 172.21.15.33:0/1622907990 learned_addr learned my addr 172.21.15.33:0/1622907990 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0effd640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 0x7f9b1006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0f7fe640 1 --2- 172.21.15.33:0/1622907990 >> v2:172.21.15.138:3300/0 conn(0x7f9b101604b0 0x7f9b10071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4268575735 0 0) 0x7f9b100cee10 con 0x7f9b1015f960 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9af8003530 con 0x7f9b1015f960 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1369923187 0 0) 0x7f9af8003530 con 0x7f9b1015f960 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 msgr2=0x7f9b1006eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 0x7f9b1006eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 >> v2:172.21.15.138:3300/0 conn(0x7f9b101604b0 msgr2=0x7f9b10071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 --2- 172.21.15.33:0/1622907990 >> v2:172.21.15.138:3300/0 conn(0x7f9b101604b0 0x7f9b10071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b100cf0f0 con 0x7f9b1015f960 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f9b04001680 con 0x7f9b1015f960 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 >> v1:172.21.15.33:6789/0 conn(0x7f9b1015f960 legacy=0x7f9b1015fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9af80039c0 0x7f9af8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 0x7f9af8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af8004d50 0x7f9af80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1006f0a0 con 0x7f9af80039c0 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9af800b9d0 con 0x7f9af8004d50 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9af800bb50 con 0x7f9b10145660 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0effd640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9af80039c0 0x7f9af8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0e7fc640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 0x7f9af8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0f7fe640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af8004d50 0x7f9af80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0effd640 1 -- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 msgr2=0x7f9af8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0effd640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b10145660 0x7f9af8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0effd640 1 -- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af8004d50 msgr2=0x7f9af80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:12.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0effd640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af8004d50 0x7f9af80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.044+0000 7f9b0effd640 1 -- 172.21.15.33:0/1622907990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af8003530 con 0x7f9af80039c0 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b0effd640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9af80039c0 0x7f9af8003de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009ae0 tx=0x7f9b0001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef95b23cdba124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000096b0 con 0x7f9af80039c0 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b000a6450 con 0x7f9af80039c0 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b0dffb640 1 -- 172.21.15.33:0/1622907990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000a6840 con 0x7f9af80039c0 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 -- 172.21.15.33:0/1622907990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9af80039c0 msgr2=0x7f9af8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 --2- 172.21.15.33:0/1622907990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9af80039c0 0x7f9af8003de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009ae0 tx=0x7f9b0001ca70 comp rx=0 tx=0).stop 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b0f7fe640 1 -- 172.21.15.33:0/1622907990 reap_dead start 2024-04-25T01:09:12.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 -- 172.21.15.33:0/1622907990 shutdown_connections 2024-04-25T01:09:12.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 -- 172.21.15.33:0/1622907990 >> 172.21.15.33:0/1622907990 conn(0x7f9b100b1810 msgr2=0x7f9b100b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:12.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 -- 172.21.15.33:0/1622907990 shutdown_connections 2024-04-25T01:09:12.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 -- 172.21.15.33:0/1622907990 wait complete. 2024-04-25T01:09:12.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 Processor -- start 2024-04-25T01:09:12.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.048+0000 7f9b15419640 1 -- start start 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80039c0 0x7f9b100aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b100ad3e0 0x7f9b10158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b100ad860 0x7f9b10159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b10072250 con 0x7f9b100ad860 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9b100720d0 con 0x7f9af80039c0 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9b100723d0 con 0x7f9b100ad3e0 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b100ad860 0x7f9b10159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b100ad860 0x7f9b10159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46804/0 (socket says 172.21.15.33:46804) 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 -- 172.21.15.33:0/259395724 learned_addr learned my addr 172.21.15.33:0/259395724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:12.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0effd640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80039c0 0x7f9b100aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0e7fc640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b100ad3e0 0x7f9b10158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 -- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b100ad3e0 msgr2=0x7f9b10158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b100ad3e0 0x7f9b10158dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 -- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80039c0 msgr2=0x7f9b100aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80039c0 0x7f9b100aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 -- 172.21.15.33:0/259395724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b100addf0 con 0x7f9b100ad860 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0effd640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80039c0 0x7f9b100aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0f7fe640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b100ad860 0x7f9b10159310 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9b10160940 tx=0x7f9b080115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9af77fe640 1 -- 172.21.15.33:0/259395724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b08011de0 con 0x7f9b100ad860 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9af77fe640 1 -- 172.21.15.33:0/259395724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0801b070 con 0x7f9b100ad860 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b100ae0d0 con 0x7f9b100ad860 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9af77fe640 1 -- 172.21.15.33:0/259395724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080039a0 con 0x7f9b100ad860 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b100ae690 con 0x7f9b100ad860 2024-04-25T01:09:12.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b10069040 con 0x7f9b100ad860 2024-04-25T01:09:12.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9af77fe640 1 -- 172.21.15.33:0/259395724 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9b08003b40 con 0x7f9b100ad860 2024-04-25T01:09:12.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9af77fe640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9afc0678f0 0x7f9afc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9af77fe640 1 -- 172.21.15.33:0/259395724 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f9b08086770 con 0x7f9b100ad860 2024-04-25T01:09:12.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.052+0000 7f9b0effd640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9afc0678f0 0x7f9afc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.056+0000 7f9b0effd640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9afc0678f0 0x7f9afc069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b000ad000 tx=0x7f9b000b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.060+0000 7f9af77fe640 1 -- 172.21.15.33:0/259395724 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9b08056eb0 con 0x7f9b100ad860 2024-04-25T01:09:12.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.216+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9b10072ac0 con 0x7f9b100ad860 2024-04-25T01:09:12.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9af77fe640 1 -- 172.21.15.33:0/259395724 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f9b080567b0 con 0x7f9b100ad860 2024-04-25T01:09:12.212 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9afc0678f0 msgr2=0x7f9afc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9afc0678f0 0x7f9afc069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9b000ad000 tx=0x7f9b000b2040 comp rx=0 tx=0).stop 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b100ad860 msgr2=0x7f9b10159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b100ad860 0x7f9b10159310 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9b10160940 tx=0x7f9b080115a0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 shutdown_connections 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9afc0678f0 0x7f9afc069db0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b100ad860 0x7f9b10159310 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9b100ad3e0 0x7f9b10158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 --2- 172.21.15.33:0/259395724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9af80039c0 0x7f9b100aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 >> 172.21.15.33:0/259395724 conn(0x7f9b100b1810 msgr2=0x7f9b10140f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 shutdown_connections 2024-04-25T01:09:12.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.220+0000 7f9b15419640 1 -- 172.21.15.33:0/259395724 wait complete. 2024-04-25T01:09:12.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:12.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.372+0000 7f0a1f50f640 1 Processor -- start 2024-04-25T01:09:12.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.372+0000 7f0a1f50f640 1 -- start start 2024-04-25T01:09:12.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1f50f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1815ae20 0x7f0a180b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1f50f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0a18072410 0x7f0a180b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1f50f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a1806e5b0 0x7f0a1806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1f50f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0a180b4a20 con 0x7f0a1806e5b0 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1f50f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0a180b2bb0 con 0x7f0a18072410 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1f50f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a180b2e90 con 0x7f0a1815ae20 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a1806e5b0 0x7f0a1806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0a1806e5b0 0x7f0a1806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46816/0 (socket says 172.21.15.33:46816) 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 -- 172.21.15.33:0/2521701593 learned_addr learned my addr 172.21.15.33:0/2521701593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1d2ab640 1 --2- 172.21.15.33:0/2521701593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1815ae20 0x7f0a180b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1caaa640 1 --2- 172.21.15.33:0/2521701593 >> v2:172.21.15.138:3300/0 conn(0x7f0a18072410 0x7f0a180b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 -- 172.21.15.33:0/2521701593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1815ae20 msgr2=0x7f0a180b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 --2- 172.21.15.33:0/2521701593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1815ae20 0x7f0a180b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 -- 172.21.15.33:0/2521701593 >> v2:172.21.15.138:3300/0 conn(0x7f0a18072410 msgr2=0x7f0a180b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:12.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 --2- 172.21.15.33:0/2521701593 >> v2:172.21.15.138:3300/0 conn(0x7f0a18072410 0x7f0a180b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 -- 172.21.15.33:0/2521701593 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a180b3170 con 0x7f0a1806e5b0 2024-04-25T01:09:12.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1d2ab640 1 --2- 172.21.15.33:0/2521701593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1815ae20 0x7f0a180b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:12.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a1daac640 1 --2- 172.21.15.33:0/2521701593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a1806e5b0 0x7f0a1806ea40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a1806cbc0 tx=0x7f0a1001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabee0bee834325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a0ffff640 1 -- 172.21.15.33:0/2521701593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a100a5780 con 0x7f0a1806e5b0 2024-04-25T01:09:12.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a0ffff640 1 -- 172.21.15.33:0/2521701593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a100a5920 con 0x7f0a1806e5b0 2024-04-25T01:09:12.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.376+0000 7f0a0ffff640 1 -- 172.21.15.33:0/2521701593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a100a5cf0 con 0x7f0a1806e5b0 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- 172.21.15.33:0/2521701593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a1806e5b0 msgr2=0x7f0a1806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/2521701593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a1806e5b0 0x7f0a1806ea40 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a1806cbc0 tx=0x7f0a1001c9f0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- 172.21.15.33:0/2521701593 shutdown_connections 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/2521701593 >> v2:172.21.15.138:3300/0 conn(0x7f0a18072410 0x7f0a180b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/2521701593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a1806e5b0 0x7f0a1806ea40 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/2521701593 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1815ae20 0x7f0a180b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- 172.21.15.33:0/2521701593 >> 172.21.15.33:0/2521701593 conn(0x7f0a180b1810 msgr2=0x7f0a180b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- 172.21.15.33:0/2521701593 shutdown_connections 2024-04-25T01:09:12.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- 172.21.15.33:0/2521701593 wait complete. 2024-04-25T01:09:12.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 Processor -- start 2024-04-25T01:09:12.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- start start 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1806e5b0 0x7f0a180a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a18072410 0x7f0a181534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1815ae20 0x7f0a18155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1815f400 con 0x7f0a18072410 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1815f280 con 0x7f0a1815ae20 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0a1815f580 con 0x7f0a1806e5b0 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a18072410 0x7f0a181534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a18072410 0x7f0a181534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46828/0 (socket says 172.21.15.33:46828) 2024-04-25T01:09:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 -- 172.21.15.33:0/792971846 learned_addr learned my addr 172.21.15.33:0/792971846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1d2ab640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1806e5b0 0x7f0a180a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1daac640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1815ae20 0x7f0a18155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 -- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1806e5b0 msgr2=0x7f0a180a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1806e5b0 0x7f0a180a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 -- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1815ae20 msgr2=0x7f0a18155740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1815ae20 0x7f0a18155740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 -- 172.21.15.33:0/792971846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a18157440 con 0x7f0a18072410 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1caaa640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a18072410 0x7f0a181534a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d800 tx=0x7f0a0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a0dffb640 1 -- 172.21.15.33:0/792971846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08018020 con 0x7f0a18072410 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a0dffb640 1 -- 172.21.15.33:0/792971846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a080096a0 con 0x7f0a18072410 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a0dffb640 1 -- 172.21.15.33:0/792971846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08009a50 con 0x7f0a18072410 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a181576e0 con 0x7f0a18072410 2024-04-25T01:09:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.380+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a18157f80 con 0x7f0a18072410 2024-04-25T01:09:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.384+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a18157ad0 con 0x7f0a18072410 2024-04-25T01:09:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.384+0000 7f0a0dffb640 1 -- 172.21.15.33:0/792971846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a0801b020 con 0x7f0a18072410 2024-04-25T01:09:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.384+0000 7f0a0dffb640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f09fc0678c0 0x7f09fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.384+0000 7f0a0dffb640 1 -- 172.21.15.33:0/792971846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0a08086570 con 0x7f0a18072410 2024-04-25T01:09:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.384+0000 7f0a1d2ab640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f09fc0678c0 0x7f09fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.384+0000 7f0a1d2ab640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f09fc0678c0 0x7f09fc069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a18165b30 tx=0x7f0a14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.388+0000 7f0a0dffb640 1 -- 172.21.15.33:0/792971846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0a08022020 con 0x7f0a18072410 2024-04-25T01:09:12.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.552+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0a18069040 con 0x7f0a18072410 2024-04-25T01:09:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.552+0000 7f0a0dffb640 1 -- 172.21.15.33:0/792971846 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f0a08056d40 con 0x7f0a18072410 2024-04-25T01:09:12.547 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f09fc0678c0 msgr2=0x7f09fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f09fc0678c0 0x7f09fc069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a18165b30 tx=0x7f0a14008040 comp rx=0 tx=0).stop 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a18072410 msgr2=0x7f0a181534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a18072410 0x7f0a181534a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d800 tx=0x7f0a0800dcc0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 shutdown_connections 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f09fc0678c0 0x7f09fc069d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a1815ae20 0x7f0a18155740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0a18072410 0x7f0a181534a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 --2- 172.21.15.33:0/792971846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0a1806e5b0 0x7f0a180a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 >> 172.21.15.33:0/792971846 conn(0x7f0a180b1810 msgr2=0x7f0a180b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 shutdown_connections 2024-04-25T01:09:12.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.556+0000 7f0a1f50f640 1 -- 172.21.15.33:0/792971846 wait complete. 2024-04-25T01:09:12.567 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4e5b12640 1 Processor -- start 2024-04-25T01:09:12.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4e5b12640 1 -- start start 2024-04-25T01:09:12.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4e5b12640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e00d1f60 0x7fc4e00d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4e5b12640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4e00b9a20 0x7fc4e00d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4e5b12640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4e00b6cc0 con 0x7fc4e015e0d0 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4e5b12640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc4e00cfb00 con 0x7fc4e00b9a20 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4e5b12640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e00b7580 con 0x7fc4e00d1f60 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4dffff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4e00b9a20 0x7fc4e00d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4dffff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc4e00b9a20 0x7fc4e00d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46838/0 (socket says 172.21.15.33:46838) 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4dffff640 1 -- 172.21.15.33:0/663439356 learned_addr learned my addr 172.21.15.33:0/663439356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4deffd640 1 --2- 172.21.15.33:0/663439356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e00d1f60 0x7fc4e00d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4dffff640 1 -- 172.21.15.33:0/663439356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e00d1f60 msgr2=0x7fc4e00d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4dffff640 1 --2- 172.21.15.33:0/663439356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e00d1f60 0x7fc4e00d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4dffff640 1 -- 172.21.15.33:0/663439356 >> v1:172.21.15.138:6789/0 conn(0x7fc4e015e0d0 legacy=0x7fc4e00cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.696+0000 7fc4dffff640 1 -- 172.21.15.33:0/663439356 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e00b6f30 con 0x7fc4e00b9a20 2024-04-25T01:09:12.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4dffff640 1 --2- 172.21.15.33:0/663439356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00d28f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009e50 tx=0x7fc4d801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6da64807c928d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4de7fc640 1 -- 172.21.15.33:0/663439356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80a58f0 con 0x7fc4e00b9a20 2024-04-25T01:09:12.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4de7fc640 1 -- 172.21.15.33:0/663439356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d80a5a90 con 0x7fc4e00b9a20 2024-04-25T01:09:12.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4de7fc640 1 -- 172.21.15.33:0/663439356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80a5e60 con 0x7fc4e00b9a20 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- 172.21.15.33:0/663439356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 msgr2=0x7fc4e00d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/663439356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00d28f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009e50 tx=0x7fc4d801ca00 comp rx=0 tx=0).stop 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- 172.21.15.33:0/663439356 shutdown_connections 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/663439356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00d28f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/663439356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e00d1f60 0x7fc4e00d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- 172.21.15.33:0/663439356 >> 172.21.15.33:0/663439356 conn(0x7fc4e00d3dd0 msgr2=0x7fc4e00d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- 172.21.15.33:0/663439356 shutdown_connections 2024-04-25T01:09:12.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- 172.21.15.33:0/663439356 wait complete. 2024-04-25T01:09:12.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 Processor -- start 2024-04-25T01:09:12.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- start start 2024-04-25T01:09:12.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4e00d1f60 0x7fc4e00b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e015e0d0 0x7fc4e00c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e016a690 con 0x7fc4e00b9a20 2024-04-25T01:09:12.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e016a510 con 0x7fc4e00d1f60 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4e5b12640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e016a810 con 0x7fc4e015e0d0 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46846/0 (socket says 172.21.15.33:46846) 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.700+0000 7fc4df7fe640 1 -- 172.21.15.33:0/3369871584 learned_addr learned my addr 172.21.15.33:0/3369871584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4deffd640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4e00d1f60 0x7fc4e00b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4dffff640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e015e0d0 0x7fc4e00c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4df7fe640 1 -- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e015e0d0 msgr2=0x7fc4e00c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4df7fe640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e015e0d0 0x7fc4e00c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4df7fe640 1 -- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4e00d1f60 msgr2=0x7fc4e00b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4df7fe640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4e00d1f60 0x7fc4e00b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4df7fe640 1 -- 172.21.15.33:0/3369871584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e00cc020 con 0x7fc4e00b9a20 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4df7fe640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00c8140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c3a0 tx=0x7fc4d000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4c7fff640 1 -- 172.21.15.33:0/3369871584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00052d0 con 0x7fc4e00b9a20 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4c7fff640 1 -- 172.21.15.33:0/3369871584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4d0005470 con 0x7fc4e00b9a20 2024-04-25T01:09:12.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4c7fff640 1 -- 172.21.15.33:0/3369871584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0007260 con 0x7fc4e00b9a20 2024-04-25T01:09:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e00c8720 con 0x7fc4e00b9a20 2024-04-25T01:09:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e00cc2e0 con 0x7fc4e00b9a20 2024-04-25T01:09:12.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e0069040 con 0x7fc4e00b9a20 2024-04-25T01:09:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4c7fff640 1 -- 172.21.15.33:0/3369871584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc4d0007400 con 0x7fc4e00b9a20 2024-04-25T01:09:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4c7fff640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4cc0678c0 0x7fc4cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4c7fff640 1 -- 172.21.15.33:0/3369871584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc4d0085460 con 0x7fc4e00b9a20 2024-04-25T01:09:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4deffd640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4cc0678c0 0x7fc4cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:12.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.704+0000 7fc4deffd640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4cc0678c0 0x7fc4cc069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4e01fa430 tx=0x7fc4d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:12.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.708+0000 7fc4c7fff640 1 -- 172.21.15.33:0/3369871584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4d0055550 con 0x7fc4e00b9a20 2024-04-25T01:09:12.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.868+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc4e00d0a80 con 0x7fc4e00b9a20 2024-04-25T01:09:12.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.868+0000 7fc4c7fff640 1 -- 172.21.15.33:0/3369871584 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc4d0059850 con 0x7fc4e00b9a20 2024-04-25T01:09:12.862 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:12.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4cc0678c0 msgr2=0x7fc4cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4cc0678c0 0x7fc4cc069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc4e01fa430 tx=0x7fc4d4008040 comp rx=0 tx=0).stop 2024-04-25T01:09:12.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 msgr2=0x7fc4e00c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:12.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00c8140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000c3a0 tx=0x7fc4d000c860 comp rx=0 tx=0).stop 2024-04-25T01:09:12.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 shutdown_connections 2024-04-25T01:09:12.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc4cc0678c0 0x7fc4cc069d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc4e015e0d0 0x7fc4e00c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4e00d1f60 0x7fc4e00b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 --2- 172.21.15.33:0/3369871584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4e00b9a20 0x7fc4e00c8140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:12.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 >> 172.21.15.33:0/3369871584 conn(0x7fc4e00d3dd0 msgr2=0x7fc4e006be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:12.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 shutdown_connections 2024-04-25T01:09:12.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:12.872+0000 7fc4e5b12640 1 -- 172.21.15.33:0/3369871584 wait complete. 2024-04-25T01:09:12.881 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.016+0000 7fc2fa600640 1 Processor -- start 2024-04-25T01:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.016+0000 7fc2fa600640 1 -- start start 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2fa600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc2f415e0a0 0x7fc2f40716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2fa600640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2f40b31a0 0x7fc2f40ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2fa600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2f415e480 con 0x7fc2f40aec20 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2fa600640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f40ae220 con 0x7fc2f40b31a0 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2fa600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc2f406e880 con 0x7fc2f415e0a0 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2f40b31a0 0x7fc2f40ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc2f40b31a0 0x7fc2f40ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46852/0 (socket says 172.21.15.33:46852) 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1042723970 learned_addr learned my addr 172.21.15.33:0/1042723970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f3fff640 1 --2- 172.21.15.33:0/1042723970 >> v2:172.21.15.138:3300/0 conn(0x7fc2f415e0a0 0x7fc2f40716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1042723970 >> v2:172.21.15.138:3300/0 conn(0x7fc2f415e0a0 msgr2=0x7fc2f40716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 --2- 172.21.15.33:0/1042723970 >> v2:172.21.15.138:3300/0 conn(0x7fc2f415e0a0 0x7fc2f40716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1042723970 >> v1:172.21.15.138:6789/0 conn(0x7fc2f40aec20 legacy=0x7fc2f4161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1042723970 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f406df00 con 0x7fc2f40b31a0 2024-04-25T01:09:13.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f37fe640 1 --2- 172.21.15.33:0/1042723970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f40ae6e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40d0b80 tx=0x7fc2e801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e08172b526b01e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:13.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f2ffd640 1 -- 172.21.15.33:0/1042723970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80b0070 con 0x7fc2f40b31a0 2024-04-25T01:09:13.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f2ffd640 1 -- 172.21.15.33:0/1042723970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e80ab600 con 0x7fc2f40b31a0 2024-04-25T01:09:13.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.020+0000 7fc2f2ffd640 1 -- 172.21.15.33:0/1042723970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80ab9d0 con 0x7fc2f40b31a0 2024-04-25T01:09:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- 172.21.15.33:0/1042723970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 msgr2=0x7fc2f40ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1042723970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f40ae6e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc2f40d0b80 tx=0x7fc2e801c450 comp rx=0 tx=0).stop 2024-04-25T01:09:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- 172.21.15.33:0/1042723970 shutdown_connections 2024-04-25T01:09:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1042723970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f40ae6e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1042723970 >> v2:172.21.15.138:3300/0 conn(0x7fc2f415e0a0 0x7fc2f40716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- 172.21.15.33:0/1042723970 >> 172.21.15.33:0/1042723970 conn(0x7fc2f40b1810 msgr2=0x7fc2f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:13.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- 172.21.15.33:0/1042723970 shutdown_connections 2024-04-25T01:09:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- 172.21.15.33:0/1042723970 wait complete. 2024-04-25T01:09:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 Processor -- start 2024-04-25T01:09:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- start start 2024-04-25T01:09:13.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f40aec20 0x7fc2f414f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f41f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2f415e0a0 0x7fc2f41525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f40b37b0 con 0x7fc2f40b31a0 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f40b3630 con 0x7fc2f40aec20 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f416af70 con 0x7fc2f415e0a0 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f41f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f41f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46866/0 (socket says 172.21.15.33:46866) 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1720399666 learned_addr learned my addr 172.21.15.33:0/1720399666 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f3fff640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f40aec20 0x7fc2f414f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f8b9d640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2f415e0a0 0x7fc2f41525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2f415e0a0 msgr2=0x7fc2f41525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2f415e0a0 0x7fc2f41525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f40aec20 msgr2=0x7fc2f414f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f40aec20 0x7fc2f414f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 -- 172.21.15.33:0/1720399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f4157350 con 0x7fc2f40b31a0 2024-04-25T01:09:13.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f3fff640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f40aec20 0x7fc2f414f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f37fe640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f41f7230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80ae040 tx=0x7fc2e801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f0ff9640 1 -- 172.21.15.33:0/1720399666 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80b0040 con 0x7fc2f40b31a0 2024-04-25T01:09:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f0ff9640 1 -- 172.21.15.33:0/1720399666 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2e80bc070 con 0x7fc2f40b31a0 2024-04-25T01:09:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f4157630 con 0x7fc2f40b31a0 2024-04-25T01:09:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2f0ff9640 1 -- 172.21.15.33:0/1720399666 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80b6880 con 0x7fc2f40b31a0 2024-04-25T01:09:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.024+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f40cdb30 con 0x7fc2f40b31a0 2024-04-25T01:09:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.028+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f4069040 con 0x7fc2f40b31a0 2024-04-25T01:09:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.028+0000 7fc2f0ff9640 1 -- 172.21.15.33:0/1720399666 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc2e80b6a20 con 0x7fc2f40b31a0 2024-04-25T01:09:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.028+0000 7fc2f0ff9640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2e00678c0 0x7fc2e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.028+0000 7fc2f0ff9640 1 -- 172.21.15.33:0/1720399666 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc2e8120600 con 0x7fc2f40b31a0 2024-04-25T01:09:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.028+0000 7fc2f3fff640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2e00678c0 0x7fc2e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.028+0000 7fc2f3fff640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2e00678c0 0x7fc2e0069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400a990 tx=0x7fc2e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:13.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.032+0000 7fc2f0ff9640 1 -- 172.21.15.33:0/1720399666 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc2e80160d0 con 0x7fc2f40b31a0 2024-04-25T01:09:13.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc2f40b2d50 con 0x7fc2f40b31a0 2024-04-25T01:09:13.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2f0ff9640 1 -- 172.21.15.33:0/1720399666 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc2e80f0d30 con 0x7fc2f40b31a0 2024-04-25T01:09:13.189 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2e00678c0 msgr2=0x7fc2e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2e00678c0 0x7fc2e0069d80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400a990 tx=0x7fc2e4008040 comp rx=0 tx=0).stop 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 msgr2=0x7fc2f41f7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f41f7230 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc2e80ae040 tx=0x7fc2e801ac90 comp rx=0 tx=0).stop 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 shutdown_connections 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc2e00678c0 0x7fc2e0069d80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc2f415e0a0 0x7fc2f41525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f40b31a0 0x7fc2f41f7230 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 --2- 172.21.15.33:0/1720399666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f40aec20 0x7fc2f414f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.196+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 >> 172.21.15.33:0/1720399666 conn(0x7fc2f40b1810 msgr2=0x7fc2f40b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.200+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 shutdown_connections 2024-04-25T01:09:13.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.200+0000 7fc2fa600640 1 -- 172.21.15.33:0/1720399666 wait complete. 2024-04-25T01:09:13.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-25T01:09:13.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.336+0000 7f1cf74f6640 1 Processor -- start 2024-04-25T01:09:13.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf74f6640 1 -- start start 2024-04-25T01:09:13.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf74f6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 0x7f1cf006e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf74f6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf00b4800 con 0x7f1cf0144730 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf74f6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf00b4ae0 con 0x7f1cf015e810 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf74f6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf00b4dc0 con 0x7f1cf015eca0 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5292640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1cf0144730 0x7f1cf00b5900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41152/0 (socket says 172.21.15.33:41152) 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5292640 1 -- 172.21.15.33:0/3938037728 learned_addr learned my addr 172.21.15.33:0/3938037728 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 0x7f1cf006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1108896849 0 0) 0x7f1cf00b4800 con 0x7f1cf0144730 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cdc003530 con 0x7f1cf0144730 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1125684078 0 0) 0x7f1cdc003530 con 0x7f1cf0144730 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 msgr2=0x7f1cf006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 0x7f1cf006e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 >> v1:172.21.15.138:6789/0 conn(0x7f1cf015e810 legacy=0x7f1cf006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf006dd50 con 0x7f1cf0144730 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f1cec001680 con 0x7f1cf0144730 2024-04-25T01:09:13.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 >> v1:172.21.15.33:6789/0 conn(0x7f1cf0144730 legacy=0x7f1cf00b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 0x7f1cdc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc003f70 0x7f1cdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc004d50 0x7f1cdc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00b4dc0 con 0x7f1cdc004d50 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc00bab0 con 0x7f1cdc003f70 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc00bcb0 con 0x7f1cf015eca0 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc004d50 0x7f1cdc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf4a91640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc003f70 0x7f1cdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5292640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 0x7f1cdc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 msgr2=0x7f1cdc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf015eca0 0x7f1cdc0039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc003f70 msgr2=0x7f1cdc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc003f70 0x7f1cdc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.340+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3938037728 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc003530 con 0x7f1cdc004d50 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc004d50 0x7f1cdc0051d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009fc0 tx=0x7f1ce801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73939057ebbb958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80b2070 con 0x7f1cdc004d50 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce80a5c90 con 0x7f1cdc004d50 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1ce7fff640 1 -- 172.21.15.33:0/3938037728 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8009c10 con 0x7f1cdc004d50 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3938037728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc004d50 msgr2=0x7f1cdc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 --2- 172.21.15.33:0/3938037728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc004d50 0x7f1cdc0051d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009fc0 tx=0x7f1ce801cc80 comp rx=0 tx=0).stop 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3938037728 reap_dead start 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3938037728 shutdown_connections 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3938037728 >> 172.21.15.33:0/3938037728 conn(0x7f1cf00b29c0 msgr2=0x7f1cf00b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3938037728 shutdown_connections 2024-04-25T01:09:13.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3938037728 wait complete. 2024-04-25T01:09:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 Processor -- start 2024-04-25T01:09:13.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- start start 2024-04-25T01:09:13.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01e54d0 0x7f1cf00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf00b0fb0 0x7f1cf01e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf01e5df0 0x7f1cf009c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf015f490 con 0x7f1cf01e5df0 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf00b50a0 con 0x7f1cf01e54d0 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf74f6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf015f610 con 0x7f1cf00b0fb0 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf01e5df0 0x7f1cf009c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf01e5df0 0x7f1cf009c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46884/0 (socket says 172.21.15.33:46884) 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3096853573 learned_addr learned my addr 172.21.15.33:0/3096853573 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf4a91640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf00b0fb0 0x7f1cf01e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5292640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01e54d0 0x7f1cf00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf00b0fb0 msgr2=0x7f1cf01e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf00b0fb0 0x7f1cf01e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01e54d0 msgr2=0x7f1cf00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01e54d0 0x7f1cf00b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf5a93640 1 -- 172.21.15.33:0/3096853573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf009acb0 con 0x7f1cf01e5df0 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.344+0000 7f1cf4a91640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf00b0fb0 0x7f1cf01e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1cf5a93640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf01e5df0 0x7f1cf009c450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8002240 tx=0x7f1ce8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1ce5ffb640 1 -- 172.21.15.33:0/3096853573 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80bd070 con 0x7f1cf01e5df0 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1ce5ffb640 1 -- 172.21.15.33:0/3096853573 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce80a5a90 con 0x7f1cf01e5df0 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1ce5ffb640 1 -- 172.21.15.33:0/3096853573 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce80b2040 con 0x7f1cf01e5df0 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf00b15c0 con 0x7f1cf01e5df0 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf00b1b80 con 0x7f1cf01e5df0 2024-04-25T01:09:13.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf0069040 con 0x7f1cf01e5df0 2024-04-25T01:09:13.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1ce5ffb640 1 -- 172.21.15.33:0/3096853573 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ce80039a0 con 0x7f1cf01e5df0 2024-04-25T01:09:13.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1ce5ffb640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cd40678c0 0x7f1cd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:13.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1ce5ffb640 1 -- 172.21.15.33:0/3096853573 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1ce8120400 con 0x7f1cf01e5df0 2024-04-25T01:09:13.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1cf5292640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cd40678c0 0x7f1cd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:13.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.348+0000 7f1cf5292640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cd40678c0 0x7f1cd4069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c10 tx=0x7f1cec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:13.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.352+0000 7f1ce5ffb640 1 -- 172.21.15.33:0/3096853573 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1ce8016990 con 0x7f1cf01e5df0 2024-04-25T01:09:13.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:13.576+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7f1cf0073130 con 0x7f1cf01e5df0 2024-04-25T01:09:14.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.336+0000 7f1ce5ffb640 1 -- 172.21.15.33:0/3096853573 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v61) ==== 123+0+0 (secure 0 0 0) 0x7f1ce80f03c0 con 0x7f1cf01e5df0 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cd40678c0 msgr2=0x7f1cd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cd40678c0 0x7f1cd4069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1cec000c10 tx=0x7f1cec009040 comp rx=0 tx=0).stop 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf01e5df0 msgr2=0x7f1cf009c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf01e5df0 0x7f1cf009c450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8002240 tx=0x7f1ce8009750 comp rx=0 tx=0).stop 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 shutdown_connections 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cd40678c0 0x7f1cd4069d80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cf01e5df0 0x7f1cf009c450 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cf00b0fb0 0x7f1cf01e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 --2- 172.21.15.33:0/3096853573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01e54d0 0x7f1cf00b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 >> 172.21.15.33:0/3096853573 conn(0x7f1cf00b29c0 msgr2=0x7f1cf015c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 shutdown_connections 2024-04-25T01:09:14.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.340+0000 7f1cf74f6640 1 -- 172.21.15.33:0/3096853573 wait complete. 2024-04-25T01:09:14.357 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T01:09:14.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 Processor -- start 2024-04-25T01:09:14.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 -- start start 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8e0c158560 0x7f8e0c0720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c06c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e0c06c760 0x7f8e0c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0c06cbb0 con 0x7f8e0c158560 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0c0b3e80 con 0x7f8e0c06c760 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e1235a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c0b4a00 con 0x7f8e0c15f100 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8e0c158560 0x7f8e0c0720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8e0c158560 0x7f8e0c0720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46890/0 (socket says 172.21.15.33:46890) 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 -- 172.21.15.33:0/2610234312 learned_addr learned my addr 172.21.15.33:0/2610234312 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e03fff640 1 --2- 172.21.15.33:0/2610234312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c06c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:14.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e108f7640 1 --2- 172.21.15.33:0/2610234312 >> v2:172.21.15.138:3300/0 conn(0x7f8e0c06c760 0x7f8e0c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 -- 172.21.15.33:0/2610234312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 msgr2=0x7f8e0c06c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 --2- 172.21.15.33:0/2610234312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c06c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 -- 172.21.15.33:0/2610234312 >> v2:172.21.15.138:3300/0 conn(0x7f8e0c06c760 msgr2=0x7f8e0c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 --2- 172.21.15.33:0/2610234312 >> v2:172.21.15.138:3300/0 conn(0x7f8e0c06c760 0x7f8e0c161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.500+0000 7f8e0bfff640 1 -- 172.21.15.33:0/2610234312 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c0b4e90 con 0x7f8e0c158560 2024-04-25T01:09:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 --2- 172.21.15.33:0/2610234312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c158560 0x7f8e0c0720f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0b3690 tx=0x7f8dfc01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4984887e1229406e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0b7fe640 1 -- 172.21.15.33:0/2610234312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0b0070 con 0x7f8e0c158560 2024-04-25T01:09:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0b7fe640 1 -- 172.21.15.33:0/2610234312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0ab600 con 0x7f8e0c158560 2024-04-25T01:09:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0b7fe640 1 -- 172.21.15.33:0/2610234312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0ab9d0 con 0x7f8e0c158560 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- 172.21.15.33:0/2610234312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c158560 msgr2=0x7f8e0c0720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 --2- 172.21.15.33:0/2610234312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c158560 0x7f8e0c0720f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0b3690 tx=0x7f8dfc01c240 comp rx=0 tx=0).stop 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- 172.21.15.33:0/2610234312 shutdown_connections 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 --2- 172.21.15.33:0/2610234312 >> v2:172.21.15.138:3300/0 conn(0x7f8e0c06c760 0x7f8e0c161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 --2- 172.21.15.33:0/2610234312 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c06c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 --2- 172.21.15.33:0/2610234312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c158560 0x7f8e0c0720f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- 172.21.15.33:0/2610234312 >> 172.21.15.33:0/2610234312 conn(0x7f8e0c0715d0 msgr2=0x7f8e0c0719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- 172.21.15.33:0/2610234312 shutdown_connections 2024-04-25T01:09:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- 172.21.15.33:0/2610234312 wait complete. 2024-04-25T01:09:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 Processor -- start 2024-04-25T01:09:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- start start 2024-04-25T01:09:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c06c760 0x7f8e0c14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e0c158560 0x7f8e0c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c0ce850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c072630 con 0x7f8e0c06c760 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0b4690 con 0x7f8e0c158560 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e1235a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c0727b0 con 0x7f8e0c15f100 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c06c760 0x7f8e0c14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c06c760 0x7f8e0c14d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46906/0 (socket says 172.21.15.33:46906) 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 -- 172.21.15.33:0/73129620 learned_addr learned my addr 172.21.15.33:0/73129620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e03fff640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e0c158560 0x7f8e0c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e108f7640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c0ce850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 -- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 msgr2=0x7f8e0c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c0ce850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 -- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e0c158560 msgr2=0x7f8e0c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e0c158560 0x7f8e0c137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e0bfff640 1 -- 172.21.15.33:0/73129620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c14ddb0 con 0x7f8e0c06c760 2024-04-25T01:09:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.504+0000 7f8e03fff640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e0c158560 0x7f8e0c137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e0bfff640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c06c760 0x7f8e0c14d870 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc01a8f0 tx=0x7f8dfc018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e097fa640 1 -- 172.21.15.33:0/73129620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0b9070 con 0x7f8e0c06c760 2024-04-25T01:09:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e097fa640 1 -- 172.21.15.33:0/73129620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0b4400 con 0x7f8e0c06c760 2024-04-25T01:09:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c14e030 con 0x7f8e0c06c760 2024-04-25T01:09:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e097fa640 1 -- 172.21.15.33:0/73129620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0b0040 con 0x7f8e0c06c760 2024-04-25T01:09:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c14e5f0 con 0x7f8e0c06c760 2024-04-25T01:09:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0c069040 con 0x7f8e0c06c760 2024-04-25T01:09:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e097fa640 1 -- 172.21.15.33:0/73129620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8dfc0bc070 con 0x7f8e0c06c760 2024-04-25T01:09:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e097fa640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8dec0678c0 0x7f8dec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e097fa640 1 -- 172.21.15.33:0/73129620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8dfc11f0b0 con 0x7f8e0c06c760 2024-04-25T01:09:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e03fff640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8dec0678c0 0x7f8dec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:14.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.508+0000 7f8e03fff640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8dec0678c0 0x7f8dec069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0ce4b0 tx=0x7f8df4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:14.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.512+0000 7f8e097fa640 1 -- 172.21.15.33:0/73129620 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8e0c14ddb0 con 0x7f8e0c06c760 2024-04-25T01:09:14.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.672+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8e0c14ecb0 con 0x7f8e0c06c760 2024-04-25T01:09:14.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.676+0000 7f8e097fa640 1 -- 172.21.15.33:0/73129620 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f8dfc0160f0 con 0x7f8e0c06c760 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8dec0678c0 msgr2=0x7f8dec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8dec0678c0 0x7f8dec069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0ce4b0 tx=0x7f8df4008040 comp rx=0 tx=0).stop 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c06c760 msgr2=0x7f8e0c14d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c06c760 0x7f8e0c14d870 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc01a8f0 tx=0x7f8dfc018fe0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 shutdown_connections 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8dec0678c0 0x7f8dec069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8e0c15f100 0x7f8e0c0ce850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e0c158560 0x7f8e0c137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 --2- 172.21.15.33:0/73129620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e0c06c760 0x7f8e0c14d870 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:14.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 >> 172.21.15.33:0/73129620 conn(0x7f8e0c0715d0 msgr2=0x7f8e0c0b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:14.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 shutdown_connections 2024-04-25T01:09:14.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:14.680+0000 7f8e1235a640 1 -- 172.21.15.33:0/73129620 wait complete. 2024-04-25T01:09:14.688 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T01:09:14.688 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T01:09:14.749 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T01:09:14.808 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T01:09:14.808 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:09:14.808 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:14.814 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:09:14.814 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:14.860 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f8edb640 1 Processor -- start 2024-04-25T01:09:15.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f8edb640 1 -- start start 2024-04-25T01:09:15.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f8edb640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f91f415c160 0x7f91f40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f8edb640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f415f9a0 con 0x7f91f40b5360 2024-04-25T01:09:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f8edb640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91f415fc80 con 0x7f91f4145470 2024-04-25T01:09:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f8edb640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f91f40b57b0 con 0x7f91f415c160 2024-04-25T01:09:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f91f40b5360 0x7f91f40cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41154/0 (socket says 172.21.15.33:41154) 2024-04-25T01:09:15.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 -- 172.21.15.33:0/3253103902 learned_addr learned my addr 172.21.15.33:0/3253103902 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f1d9b640 1 --2- 172.21.15.33:0/3253103902 >> v2:172.21.15.138:3300/0 conn(0x7f91f415c160 0x7f91f40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413213687 0 0) 0x7f91f415f9a0 con 0x7f91f40b5360 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91e0003530 con 0x7f91f40b5360 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 44330435 0 0) 0x7f91e0003530 con 0x7f91f40b5360 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 >> v2:172.21.15.138:3300/0 conn(0x7f91f415c160 msgr2=0x7f91f40b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 --2- 172.21.15.33:0/3253103902 >> v2:172.21.15.138:3300/0 conn(0x7f91f415c160 0x7f91f40b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 >> v1:172.21.15.138:6789/0 conn(0x7f91f4145470 legacy=0x7f91f40cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f415ff60 con 0x7f91f40b5360 2024-04-25T01:09:15.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f91ec001a00 con 0x7f91f40b5360 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 >> v1:172.21.15.33:6789/0 conn(0x7f91f40b5360 legacy=0x7f91f40cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00039c0 0x7f91e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91e0004810 0x7f91e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e00051d0 0x7f91e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91f40b57b0 con 0x7f91e00051d0 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91e000c120 con 0x7f91e0004810 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91e000c320 con 0x7f91e00039c0 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e00051d0 0x7f91e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f1d9b640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91e0004810 0x7f91e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f259c640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00039c0 0x7f91e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 -- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00039c0 msgr2=0x7f91e0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00039c0 0x7f91e0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 -- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91e0004810 msgr2=0x7f91e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91e0004810 0x7f91e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.048+0000 7f91f2d9d640 1 -- 172.21.15.33:0/3253103902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e0003530 con 0x7f91e00051d0 2024-04-25T01:09:15.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f2d9d640 1 -- 172.21.15.33:0/3253103902 reap_dead start 2024-04-25T01:09:15.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f2d9d640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e00051d0 0x7f91e000b9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00a840 tx=0x7f91ec01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e16404df57dcb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec00a460 con 0x7f91e00051d0 2024-04-25T01:09:15.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ec0ac640 con 0x7f91e00051d0 2024-04-25T01:09:15.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f159a640 1 -- 172.21.15.33:0/3253103902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0aca30 con 0x7f91e00051d0 2024-04-25T01:09:15.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 -- 172.21.15.33:0/3253103902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e00051d0 msgr2=0x7f91e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e00051d0 0x7f91e000b9d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f91ec00a840 tx=0x7f91ec01d360 comp rx=0 tx=0).stop 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 -- 172.21.15.33:0/3253103902 shutdown_connections 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 --2- 172.21.15.33:0/3253103902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91e00051d0 0x7f91e000b9d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 -- 172.21.15.33:0/3253103902 >> 172.21.15.33:0/3253103902 conn(0x7f91f40b1810 msgr2=0x7f91f40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 -- 172.21.15.33:0/3253103902 shutdown_connections 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 -- 172.21.15.33:0/3253103902 wait complete. 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 Processor -- start 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.052+0000 7f91f8edb640 1 -- start start 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00051d0 0x7f91f415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f40ab040 0x7f91f40a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f40a9970 0x7f91f413efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91f4145850 con 0x7f91f40a9970 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91f415f750 con 0x7f91f40ab040 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f91f406e850 con 0x7f91e00051d0 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f40a9970 0x7f91f413efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f40a9970 0x7f91f413efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46930/0 (socket says 172.21.15.33:46930) 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 -- 172.21.15.33:0/2478018783 learned_addr learned my addr 172.21.15.33:0/2478018783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f1d9b640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f40ab040 0x7f91f40a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f259c640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00051d0 0x7f91f415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 -- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00051d0 msgr2=0x7f91f415b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00051d0 0x7f91f415b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 -- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f40ab040 msgr2=0x7f91f40a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f40ab040 0x7f91f40a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 -- 172.21.15.33:0/2478018783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f40a9fa0 con 0x7f91f40a9970 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f259c640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00051d0 0x7f91f415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f1d9b640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f40ab040 0x7f91f40a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f2d9d640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f40a9970 0x7f91f413efe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f91ec01d190 tx=0x7f91e0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91eaffd640 1 -- 172.21.15.33:0/2478018783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0b0030 con 0x7f91f40a9970 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91eaffd640 1 -- 172.21.15.33:0/2478018783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91ec0019c0 con 0x7f91f40a9970 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f40aa280 con 0x7f91f40a9970 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91eaffd640 1 -- 172.21.15.33:0/2478018783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ec0ac9f0 con 0x7f91f40a9970 2024-04-25T01:09:15.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f40aa840 con 0x7f91f40a9970 2024-04-25T01:09:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91f4069040 con 0x7f91f40a9970 2024-04-25T01:09:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91eaffd640 1 -- 172.21.15.33:0/2478018783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f91ec0acb90 con 0x7f91f40a9970 2024-04-25T01:09:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91eaffd640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91d40678c0 0x7f91d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91eaffd640 1 -- 172.21.15.33:0/2478018783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f91ec120000 con 0x7f91f40a9970 2024-04-25T01:09:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.056+0000 7f91f259c640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91d40678c0 0x7f91d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.060+0000 7f91f259c640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91d40678c0 0x7f91d4069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91e4007fd0 tx=0x7f91e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.060+0000 7f91eaffd640 1 -- 172.21.15.33:0/2478018783 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f91ec017070 con 0x7f91f40a9970 2024-04-25T01:09:15.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.224+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f91f4140a50 con 0x7f91f40a9970 2024-04-25T01:09:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91eaffd640 1 -- 172.21.15.33:0/2478018783 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7f91ec0f06b0 con 0x7f91f40a9970 2024-04-25T01:09:15.220 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:15.221 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:15.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91d40678c0 msgr2=0x7f91d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91d40678c0 0x7f91d4069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91e4007fd0 tx=0x7f91e4009040 comp rx=0 tx=0).stop 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f40a9970 msgr2=0x7f91f413efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f40a9970 0x7f91f413efe0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f91ec01d190 tx=0x7f91e0005840 comp rx=0 tx=0).stop 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 shutdown_connections 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91d40678c0 0x7f91d4069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91f40a9970 0x7f91f413efe0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91f40ab040 0x7f91f40a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 --2- 172.21.15.33:0/2478018783 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f91e00051d0 0x7f91f415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 >> 172.21.15.33:0/2478018783 conn(0x7f91f40b1810 msgr2=0x7f91f40cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 shutdown_connections 2024-04-25T01:09:15.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.228+0000 7f91f8edb640 1 -- 172.21.15.33:0/2478018783 wait complete. 2024-04-25T01:09:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:15.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cf32f640 1 Processor -- start 2024-04-25T01:09:15.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cf32f640 1 -- start start 2024-04-25T01:09:15.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cf32f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f51c80b5580 0x7f51c80b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cf32f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 0x7f51c815da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cf32f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51c80b05a0 con 0x7f51c80b50d0 2024-04-25T01:09:15.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cf32f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f51c80b5ea0 con 0x7f51c80b5580 2024-04-25T01:09:15.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cf32f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51c80af500 con 0x7f51c815d5d0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cc8ca640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f51c80b50d0 0x7f51c815ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41168/0 (socket says 172.21.15.33:41168) 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cc8ca640 1 -- 172.21.15.33:0/1672880377 learned_addr learned my addr 172.21.15.33:0/1672880377 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672880377 >> v2:172.21.15.138:3300/0 conn(0x7f51c80b5580 0x7f51c80b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd8cc640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 0x7f51c815da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1746403873 0 0) 0x7f51b8001120 con 0x7f51c80b50d0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51b4003530 con 0x7f51c80b50d0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2176549985 0 0) 0x7f51b4003530 con 0x7f51c80b50d0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 msgr2=0x7f51c815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 0x7f51c815da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 >> v2:172.21.15.138:3300/0 conn(0x7f51c80b5580 msgr2=0x7f51c80b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 --2- 172.21.15.33:0/1672880377 >> v2:172.21.15.138:3300/0 conn(0x7f51c80b5580 0x7f51c80b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c80af7e0 con 0x7f51c80b50d0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f51b8001850 con 0x7f51c80b50d0 2024-04-25T01:09:15.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 >> v1:172.21.15.33:6789/0 conn(0x7f51c80b50d0 legacy=0x7f51c815ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b40039c0 0x7f51b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 0x7f51b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b4004d50 0x7f51b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80b5ea0 con 0x7f51b40039c0 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80af500 con 0x7f51b4004d50 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51b400b9d0 con 0x7f51c815d5d0 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b40039c0 0x7f51b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cc8ca640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 0x7f51b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd0cb640 1 -- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 msgr2=0x7f51b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd8cc640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b4004d50 0x7f51b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51c815d5d0 0x7f51b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd0cb640 1 -- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b4004d50 msgr2=0x7f51b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b4004d50 0x7f51b40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.376+0000 7f51cd0cb640 1 -- 172.21.15.33:0/1672880377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b4003530 con 0x7f51b40039c0 2024-04-25T01:09:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b40039c0 0x7f51b4003de0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009ae0 tx=0x7f51c401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b7167d3f6f2538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c40096b0 con 0x7f51b40039c0 2024-04-25T01:09:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c40a6470 con 0x7f51b40039c0 2024-04-25T01:09:15.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51bffff640 1 -- 172.21.15.33:0/1672880377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c40a6860 con 0x7f51b40039c0 2024-04-25T01:09:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672880377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b40039c0 msgr2=0x7f51b4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 --2- 172.21.15.33:0/1672880377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51b40039c0 0x7f51b4003de0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009ae0 tx=0x7f51c401cab0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cd8cc640 1 -- 172.21.15.33:0/1672880377 reap_dead start 2024-04-25T01:09:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672880377 shutdown_connections 2024-04-25T01:09:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672880377 >> 172.21.15.33:0/1672880377 conn(0x7f51c809a9d0 msgr2=0x7f51c809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:15.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672880377 shutdown_connections 2024-04-25T01:09:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672880377 wait complete. 2024-04-25T01:09:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 Processor -- start 2024-04-25T01:09:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.380+0000 7f51cf32f640 1 -- start start 2024-04-25T01:09:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40039c0 0x7f51c81521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8156310 0x7f51c8135bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c8132c00 0x7f51c8133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f51c815e1f0 con 0x7f51c8132c00 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51c80af1b0 con 0x7f51c8156310 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f51c815e370 con 0x7f51b40039c0 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c8132c00 0x7f51c8133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c8132c00 0x7f51c8133080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46952/0 (socket says 172.21.15.33:46952) 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 -- 172.21.15.33:0/1672277140 learned_addr learned my addr 172.21.15.33:0/1672277140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cc8ca640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8156310 0x7f51c8135bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40039c0 0x7f51c81521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 -- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40039c0 msgr2=0x7f51c81521c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40039c0 0x7f51c81521c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 -- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8156310 msgr2=0x7f51c8135bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8156310 0x7f51c8135bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 -- 172.21.15.33:0/1672277140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c81596b0 con 0x7f51c8132c00 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40039c0 0x7f51c81521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cc8ca640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8156310 0x7f51c8135bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:15.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd8cc640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c8132c00 0x7f51c8133080 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f51c80b4830 tx=0x7f51c000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51bdffb640 1 -- 172.21.15.33:0/1672277140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c001c070 con 0x7f51c8132c00 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51bdffb640 1 -- 172.21.15.33:0/1672277140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51c0017400 con 0x7f51c8132c00 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c81335c0 con 0x7f51c8132c00 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51bdffb640 1 -- 172.21.15.33:0/1672277140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c00177f0 con 0x7f51c8132c00 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c8133d20 con 0x7f51c8132c00 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c8069040 con 0x7f51c8132c00 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51bdffb640 1 -- 172.21.15.33:0/1672277140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f51c001f070 con 0x7f51c8132c00 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51bdffb640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f51ac0678f0 0x7f51ac069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51bdffb640 1 -- 172.21.15.33:0/1672277140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f51c0086240 con 0x7f51c8132c00 2024-04-25T01:09:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.384+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f51ac0678f0 0x7f51ac069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.388+0000 7f51cd0cb640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f51ac0678f0 0x7f51ac069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51c40ad000 tx=0x7f51c40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.388+0000 7f51bdffb640 1 -- 172.21.15.33:0/1672277140 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f51c0012070 con 0x7f51c8132c00 2024-04-25T01:09:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.556+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51c80d20f0 con 0x7f51c8132c00 2024-04-25T01:09:15.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.556+0000 7f51bdffb640 1 -- 172.21.15.33:0/1672277140 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f51c0056980 con 0x7f51c8132c00 2024-04-25T01:09:15.549 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:15.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f51ac0678f0 msgr2=0x7f51ac069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f51ac0678f0 0x7f51ac069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f51c40ad000 tx=0x7f51c40046d0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c8132c00 msgr2=0x7f51c8133080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c8132c00 0x7f51c8133080 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f51c80b4830 tx=0x7f51c000f4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 shutdown_connections 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f51ac0678f0 0x7f51ac069db0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51c8132c00 0x7f51c8133080 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51c8156310 0x7f51c8135bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 --2- 172.21.15.33:0/1672277140 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f51b40039c0 0x7f51c81521c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 >> 172.21.15.33:0/1672277140 conn(0x7f51c809a9d0 msgr2=0x7f51c8150950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 shutdown_connections 2024-04-25T01:09:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.560+0000 7f51cf32f640 1 -- 172.21.15.33:0/1672277140 wait complete. 2024-04-25T01:09:15.570 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd7eee0640 1 Processor -- start 2024-04-25T01:09:15.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd7eee0640 1 -- start start 2024-04-25T01:09:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd7eee0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcd78072410 0x7fcd780b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd7eee0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd780b4a20 con 0x7fcd7806ecc0 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd7eee0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd780b2bb0 con 0x7fcd7815ae20 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd7eee0640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fcd780b2e90 con 0x7fcd78072410 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd77fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcd7806ecc0 0x7fcd780cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41176/0 (socket says 172.21.15.33:41176) 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd77fff640 1 -- 172.21.15.33:0/3221970335 learned_addr learned my addr 172.21.15.33:0/3221970335 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.708+0000 7fcd7d47d640 1 --2- 172.21.15.33:0/3221970335 >> v2:172.21.15.33:3300/0 conn(0x7fcd78072410 0x7fcd780b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd777fe640 1 -- 172.21.15.33:0/3221970335 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2119673475 0 0) 0x7fcd780b4a20 con 0x7fcd7806ecc0 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd777fe640 1 -- 172.21.15.33:0/3221970335 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd60003530 con 0x7fcd7806ecc0 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7d47d640 1 -- 172.21.15.33:0/3221970335 >> v1:172.21.15.138:6789/0 conn(0x7fcd7815ae20 legacy=0x7fcd7806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7d47d640 1 -- 172.21.15.33:0/3221970335 >> v1:172.21.15.33:6789/0 conn(0x7fcd7806ecc0 legacy=0x7fcd780cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7d47d640 1 -- 172.21.15.33:0/3221970335 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd780b3170 con 0x7fcd78072410 2024-04-25T01:09:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7d47d640 1 --2- 172.21.15.33:0/3221970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd780b3690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009d90 tx=0x7fcd7001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b58542fda9da60c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd777fe640 1 -- 172.21.15.33:0/3221970335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700b2070 con 0x7fcd78072410 2024-04-25T01:09:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd777fe640 1 -- 172.21.15.33:0/3221970335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd700a59f0 con 0x7fcd78072410 2024-04-25T01:09:15.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd777fe640 1 -- 172.21.15.33:0/3221970335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd700a5da0 con 0x7fcd78072410 2024-04-25T01:09:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7eee0640 1 -- 172.21.15.33:0/3221970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 msgr2=0x7fcd780b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/3221970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd780b3690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009d90 tx=0x7fcd7001c930 comp rx=0 tx=0).stop 2024-04-25T01:09:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7eee0640 1 -- 172.21.15.33:0/3221970335 shutdown_connections 2024-04-25T01:09:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/3221970335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd780b3690 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7eee0640 1 -- 172.21.15.33:0/3221970335 >> 172.21.15.33:0/3221970335 conn(0x7fcd780b1810 msgr2=0x7fcd780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7eee0640 1 -- 172.21.15.33:0/3221970335 shutdown_connections 2024-04-25T01:09:15.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.712+0000 7fcd7eee0640 1 -- 172.21.15.33:0/3221970335 wait complete. 2024-04-25T01:09:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 Processor -- start 2024-04-25T01:09:15.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 -- start start 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd7806ecc0 0x7fcd7813c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd78169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd7815ae20 0x7fcd7815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7806d620 con 0x7fcd78072410 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7806d4a0 con 0x7fcd7815ae20 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7806d7a0 con 0x7fcd7806ecc0 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd78169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd78169630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46978/0 (socket says 172.21.15.33:46978) 2024-04-25T01:09:15.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 -- 172.21.15.33:0/2272237103 learned_addr learned my addr 172.21.15.33:0/2272237103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7d47d640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd7815ae20 0x7fcd7815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7cc7c640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd7806ecc0 0x7fcd7813c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 -- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd7806ecc0 msgr2=0x7fcd7813c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd7806ecc0 0x7fcd7813c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 -- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd7815ae20 msgr2=0x7fcd7815bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd7815ae20 0x7fcd7815bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 -- 172.21.15.33:0/2272237103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd78169da0 con 0x7fcd78072410 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7d47d640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd7815ae20 0x7fcd7815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7cc7c640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd7806ecc0 0x7fcd7813c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd77fff640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd78169630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c450 tx=0x7fcd6c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd757fa640 1 -- 172.21.15.33:0/2272237103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c017070 con 0x7fcd78072410 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd757fa640 1 -- 172.21.15.33:0/2272237103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd6c005cf0 con 0x7fcd78072410 2024-04-25T01:09:15.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7816a080 con 0x7fcd78072410 2024-04-25T01:09:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd757fa640 1 -- 172.21.15.33:0/2272237103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c007290 con 0x7fcd78072410 2024-04-25T01:09:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd78158c90 con 0x7fcd78072410 2024-04-25T01:09:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7816a5a0 con 0x7fcd78072410 2024-04-25T01:09:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.716+0000 7fcd757fa640 1 -- 172.21.15.33:0/2272237103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd6c01a070 con 0x7fcd78072410 2024-04-25T01:09:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.720+0000 7fcd757fa640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcd640678c0 0x7fcd64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.720+0000 7fcd757fa640 1 -- 172.21.15.33:0/2272237103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcd6c085450 con 0x7fcd78072410 2024-04-25T01:09:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.720+0000 7fcd7cc7c640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcd640678c0 0x7fcd64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:15.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.720+0000 7fcd7cc7c640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcd640678c0 0x7fcd64069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd680098b0 tx=0x7fcd68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:15.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.724+0000 7fcd757fa640 1 -- 172.21.15.33:0/2272237103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcd6c055b80 con 0x7fcd78072410 2024-04-25T01:09:15.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcd78069040 con 0x7fcd78072410 2024-04-25T01:09:15.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd757fa640 1 -- 172.21.15.33:0/2272237103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fcd6c055480 con 0x7fcd78072410 2024-04-25T01:09:15.877 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcd640678c0 msgr2=0x7fcd64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcd640678c0 0x7fcd64069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcd680098b0 tx=0x7fcd68008040 comp rx=0 tx=0).stop 2024-04-25T01:09:15.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 msgr2=0x7fcd78169630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd78169630 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00c450 tx=0x7fcd6c00c910 comp rx=0 tx=0).stop 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 shutdown_connections 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcd640678c0 0x7fcd64069d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd7815ae20 0x7fcd7815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd78072410 0x7fcd78169630 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 --2- 172.21.15.33:0/2272237103 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcd7806ecc0 0x7fcd7813c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 >> 172.21.15.33:0/2272237103 conn(0x7fcd780b1810 msgr2=0x7fcd78071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 shutdown_connections 2024-04-25T01:09:15.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:15.884+0000 7fcd7eee0640 1 -- 172.21.15.33:0/2272237103 wait complete. 2024-04-25T01:09:15.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc40af58640 1 Processor -- start 2024-04-25T01:09:16.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc40af58640 1 -- start start 2024-04-25T01:09:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc40af58640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc4040d2ac0 0x7fc404168d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc40af58640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4040b6cc0 con 0x7fc4040d1f60 2024-04-25T01:09:16.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc40af58640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4040b6f30 con 0x7fc40415e0d0 2024-04-25T01:09:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc40af58640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc4040cfb00 con 0x7fc4040d2ac0 2024-04-25T01:09:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc4040d1f60 0x7fc4040d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41184/0 (socket says 172.21.15.33:41184) 2024-04-25T01:09:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 -- 172.21.15.33:0/2804452315 learned_addr learned my addr 172.21.15.33:0/2804452315 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:16.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4094f5640 1 --2- 172.21.15.33:0/2804452315 >> v2:172.21.15.138:3300/0 conn(0x7fc4040d2ac0 0x7fc404168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1513711208 0 0) 0x7fc4040b6cc0 con 0x7fc4040d1f60 2024-04-25T01:09:16.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3ec003530 con 0x7fc4040d1f60 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 619400998 0 0) 0x7fc3ec003530 con 0x7fc4040d1f60 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 >> v2:172.21.15.138:3300/0 conn(0x7fc4040d2ac0 msgr2=0x7fc404168d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 --2- 172.21.15.33:0/2804452315 >> v2:172.21.15.138:3300/0 conn(0x7fc4040d2ac0 0x7fc404168d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 >> v1:172.21.15.138:6789/0 conn(0x7fc40415e0d0 legacy=0x7fc4040cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4040b7210 con 0x7fc4040d1f60 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fc3f8001680 con 0x7fc4040d1f60 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 >> v1:172.21.15.33:6789/0 conn(0x7fc4040d1f60 legacy=0x7fc4040d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec0039c0 0x7fc3ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3ec004810 0x7fc3ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec0051d0 0x7fc3ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040cfb00 con 0x7fc3ec004810 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3ec00c070 con 0x7fc3ec0051d0 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3ec00c270 con 0x7fc3ec0039c0 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3ec004810 0x7fc3ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4094f5640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec0051d0 0x7fc3ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc408cf4640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec0039c0 0x7fc3ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 -- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec0039c0 msgr2=0x7fc3ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec0039c0 0x7fc3ec003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 -- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec0051d0 msgr2=0x7fc3ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec0051d0 0x7fc3ec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc403fff640 1 -- 172.21.15.33:0/2804452315 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3ec003530 con 0x7fc3ec004810 2024-04-25T01:09:16.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4094f5640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec0051d0 0x7fc3ec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc408cf4640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec0039c0 0x7fc3ec003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:16.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.032+0000 7fc4094f5640 1 -- 172.21.15.33:0/2804452315 reap_dead start 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc403fff640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3ec004810 0x7fc3ec004c90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c210 tx=0x7fc3f801f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f212d0681a194081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8007d20 con 0x7fc3ec004810 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f8005ae0 con 0x7fc3ec004810 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc4037fe640 1 -- 172.21.15.33:0/2804452315 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8007070 con 0x7fc3ec004810 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 -- 172.21.15.33:0/2804452315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3ec004810 msgr2=0x7fc3ec004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3ec004810 0x7fc3ec004c90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c210 tx=0x7fc3f801f330 comp rx=0 tx=0).stop 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 -- 172.21.15.33:0/2804452315 shutdown_connections 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 --2- 172.21.15.33:0/2804452315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3ec004810 0x7fc3ec004c90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 -- 172.21.15.33:0/2804452315 >> 172.21.15.33:0/2804452315 conn(0x7fc4040d3dd0 msgr2=0x7fc4040d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 -- 172.21.15.33:0/2804452315 shutdown_connections 2024-04-25T01:09:16.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 -- 172.21.15.33:0/2804452315 wait complete. 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 Processor -- start 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.036+0000 7fc40af58640 1 -- start start 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec004810 0x7fc4040caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc404160a30 0x7fc4040cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc40414cdf0 0x7fc40414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc40415e4b0 con 0x7fc404160a30 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc4040b7580 con 0x7fc40414cdf0 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc40415e630 con 0x7fc3ec004810 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc404160a30 0x7fc4040cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc404160a30 0x7fc4040cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46990/0 (socket says 172.21.15.33:46990) 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 -- 172.21.15.33:0/2219983840 learned_addr learned my addr 172.21.15.33:0/2219983840 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc408cf4640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec004810 0x7fc4040caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4094f5640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc40414cdf0 0x7fc40414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 -- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec004810 msgr2=0x7fc4040caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec004810 0x7fc4040caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 -- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc40414cdf0 msgr2=0x7fc40414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc40414cdf0 0x7fc40414d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 -- 172.21.15.33:0/2219983840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc404160e90 con 0x7fc404160a30 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4094f5640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc40414cdf0 0x7fc40414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc403fff640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc404160a30 0x7fc4040cd9a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c210 tx=0x7fc3ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4017fa640 1 -- 172.21.15.33:0/2219983840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8007d20 con 0x7fc404160a30 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4017fa640 1 -- 172.21.15.33:0/2219983840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f80b6070 con 0x7fc404160a30 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4040b93e0 con 0x7fc404160a30 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4017fa640 1 -- 172.21.15.33:0/2219983840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f80047f0 con 0x7fc404160a30 2024-04-25T01:09:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40414ece0 con 0x7fc404160a30 2024-04-25T01:09:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4040b9900 con 0x7fc404160a30 2024-04-25T01:09:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4017fa640 1 -- 172.21.15.33:0/2219983840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3f80b8020 con 0x7fc404160a30 2024-04-25T01:09:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4017fa640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3f00678c0 0x7fc3f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc4017fa640 1 -- 172.21.15.33:0/2219983840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc3f8123550 con 0x7fc404160a30 2024-04-25T01:09:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.040+0000 7fc408cf4640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3f00678c0 0x7fc3f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.044+0000 7fc408cf4640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3f00678c0 0x7fc3f0069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000c10 tx=0x7fc3f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.044+0000 7fc4017fa640 1 -- 172.21.15.33:0/2219983840 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc4040b9900 con 0x7fc404160a30 2024-04-25T01:09:16.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.208+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc404069040 con 0x7fc404160a30 2024-04-25T01:09:16.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.208+0000 7fc4017fa640 1 -- 172.21.15.33:0/2219983840 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fc3f8019040 con 0x7fc404160a30 2024-04-25T01:09:16.201 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:16.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.208+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3f00678c0 msgr2=0x7fc3f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.208+0000 7fc40af58640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3f00678c0 0x7fc3f0069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4000c10 tx=0x7fc3f4009040 comp rx=0 tx=0).stop 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.208+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc404160a30 msgr2=0x7fc4040cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.208+0000 7fc40af58640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc404160a30 0x7fc4040cd9a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c210 tx=0x7fc3ec005840 comp rx=0 tx=0).stop 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 shutdown_connections 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3f00678c0 0x7fc3f0069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc40414cdf0 0x7fc40414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc404160a30 0x7fc4040cd9a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 --2- 172.21.15.33:0/2219983840 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3ec004810 0x7fc4040caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 >> 172.21.15.33:0/2219983840 conn(0x7fc4040d3dd0 msgr2=0x7fc4040b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 shutdown_connections 2024-04-25T01:09:16.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.212+0000 7fc40af58640 1 -- 172.21.15.33:0/2219983840 wait complete. 2024-04-25T01:09:16.221 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:16.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddc0c31640 1 Processor -- start 2024-04-25T01:09:16.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddc0c31640 1 -- start start 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddc0c31640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fddbc0d1f60 0x7fddbc0d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddc0c31640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fddbc0b9a20 0x7fddbc0d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddc0c31640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddbc0b6cc0 con 0x7fddbc15e0d0 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddc0c31640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc0cfb00 con 0x7fddbc0b9a20 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddc0c31640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc0b7580 con 0x7fddbc0d1f60 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddbad9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fddbc0b9a20 0x7fddbc0d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddbad9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fddbc0b9a20 0x7fddbc0d28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46992/0 (socket says 172.21.15.33:46992) 2024-04-25T01:09:16.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddbad9d640 1 -- 172.21.15.33:0/217019413 learned_addr learned my addr 172.21.15.33:0/217019413 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddb1d9b640 1 --2- 172.21.15.33:0/217019413 >> v2:172.21.15.138:3300/0 conn(0x7fddbc0d1f60 0x7fddbc0d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddbad9d640 1 -- 172.21.15.33:0/217019413 >> v2:172.21.15.138:3300/0 conn(0x7fddbc0d1f60 msgr2=0x7fddbc0d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddbad9d640 1 --2- 172.21.15.33:0/217019413 >> v2:172.21.15.138:3300/0 conn(0x7fddbc0d1f60 0x7fddbc0d23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddbad9d640 1 -- 172.21.15.33:0/217019413 >> v1:172.21.15.138:6789/0 conn(0x7fddbc15e0d0 legacy=0x7fddbc0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.352+0000 7fddbad9d640 1 -- 172.21.15.33:0/217019413 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc0b6f30 con 0x7fddbc0b9a20 2024-04-25T01:09:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddbad9d640 1 --2- 172.21.15.33:0/217019413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc0d28f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fddbc08e300 tx=0x7fddb401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1aed3cdd329e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddb9d9b640 1 -- 172.21.15.33:0/217019413 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40a58a0 con 0x7fddbc0b9a20 2024-04-25T01:09:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddb9d9b640 1 -- 172.21.15.33:0/217019413 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb40a5a40 con 0x7fddbc0b9a20 2024-04-25T01:09:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddb9d9b640 1 -- 172.21.15.33:0/217019413 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb40a5e10 con 0x7fddbc0b9a20 2024-04-25T01:09:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- 172.21.15.33:0/217019413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 msgr2=0x7fddbc0d28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 --2- 172.21.15.33:0/217019413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc0d28f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fddbc08e300 tx=0x7fddb401c9c0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- 172.21.15.33:0/217019413 shutdown_connections 2024-04-25T01:09:16.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 --2- 172.21.15.33:0/217019413 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc0d28f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 --2- 172.21.15.33:0/217019413 >> v2:172.21.15.138:3300/0 conn(0x7fddbc0d1f60 0x7fddbc0d23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- 172.21.15.33:0/217019413 >> 172.21.15.33:0/217019413 conn(0x7fddbc0d3dd0 msgr2=0x7fddbc0d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:16.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- 172.21.15.33:0/217019413 shutdown_connections 2024-04-25T01:09:16.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- 172.21.15.33:0/217019413 wait complete. 2024-04-25T01:09:16.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 Processor -- start 2024-04-25T01:09:16.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- start start 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddbc0d1f60 0x7fddbc0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddbc15e0d0 0x7fddbc15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc16a690 con 0x7fddbc0b9a20 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc16a510 con 0x7fddbc15e0d0 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddc0c31640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc16a810 con 0x7fddbc0d1f60 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddba59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddba59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47002/0 (socket says 172.21.15.33:47002) 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddba59c640 1 -- 172.21.15.33:0/1560869848 learned_addr learned my addr 172.21.15.33:0/1560869848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddb1d9b640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddbc0d1f60 0x7fddbc0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.356+0000 7fddbad9d640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddbc15e0d0 0x7fddbc15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddba59c640 1 -- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddbc0d1f60 msgr2=0x7fddbc0cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddba59c640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddbc0d1f60 0x7fddbc0cd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddba59c640 1 -- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddbc15e0d0 msgr2=0x7fddbc15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddba59c640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddbc15e0d0 0x7fddbc15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddba59c640 1 -- 172.21.15.33:0/1560869848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc0c7860 con 0x7fddbc0b9a20 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddba59c640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc159770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c3a0 tx=0x7fddac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddb37fe640 1 -- 172.21.15.33:0/1560869848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac0052d0 con 0x7fddbc0b9a20 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddb37fe640 1 -- 172.21.15.33:0/1560869848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddac005470 con 0x7fddbc0b9a20 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc0c7b40 con 0x7fddbc0b9a20 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddb37fe640 1 -- 172.21.15.33:0/1560869848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddac007260 con 0x7fddbc0b9a20 2024-04-25T01:09:16.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc0c8100 con 0x7fddbc0b9a20 2024-04-25T01:09:16.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc069040 con 0x7fddbc0b9a20 2024-04-25T01:09:16.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddb37fe640 1 -- 172.21.15.33:0/1560869848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fddac007400 con 0x7fddbc0b9a20 2024-04-25T01:09:16.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddb37fe640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd9c0678c0 0x7fdd9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddb37fe640 1 -- 172.21.15.33:0/1560869848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fddac085b90 con 0x7fddbc0b9a20 2024-04-25T01:09:16.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.360+0000 7fddb1d9b640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd9c0678c0 0x7fdd9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.364+0000 7fddb1d9b640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd9c0678c0 0x7fdd9c069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0cacd0 tx=0x7fdda4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.364+0000 7fddb37fe640 1 -- 172.21.15.33:0/1560869848 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fddac055df0 con 0x7fddbc0b9a20 2024-04-25T01:09:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.520+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fddbc073cd0 con 0x7fddbc0b9a20 2024-04-25T01:09:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.520+0000 7fddb37fe640 1 -- 172.21.15.33:0/1560869848 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fddac05a0f0 con 0x7fddbc0b9a20 2024-04-25T01:09:16.514 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd9c0678c0 msgr2=0x7fdd9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd9c0678c0 0x7fdd9c069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fddbc0cacd0 tx=0x7fdda4008040 comp rx=0 tx=0).stop 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 msgr2=0x7fddbc159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc159770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fddac00c3a0 tx=0x7fddac00c860 comp rx=0 tx=0).stop 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 shutdown_connections 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdd9c0678c0 0x7fdd9c069d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddbc15e0d0 0x7fddbc15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddbc0d1f60 0x7fddbc0cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 --2- 172.21.15.33:0/1560869848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddbc0b9a20 0x7fddbc159770 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 >> 172.21.15.33:0/1560869848 conn(0x7fddbc0d3dd0 msgr2=0x7fddbc0b9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 shutdown_connections 2024-04-25T01:09:16.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.524+0000 7fddc0c31640 1 -- 172.21.15.33:0/1560869848 wait complete. 2024-04-25T01:09:16.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:16.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b662d4640 1 Processor -- start 2024-04-25T01:09:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b662d4640 1 -- start start 2024-04-25T01:09:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b662d4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b60072540 0x7f5b601659f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b662d4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b601448d0 con 0x7f5b601444f0 2024-04-25T01:09:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b662d4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b6015ec20 con 0x7f5b6015e660 2024-04-25T01:09:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b662d4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f5b600ce0c0 con 0x7f5b60072540 2024-04-25T01:09:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b64871640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5b60072540 0x7f5b601659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b5ffff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f5b601444f0 0x7f5b600b5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41194/0 (socket says 172.21.15.33:41194) 2024-04-25T01:09:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b5ffff640 1 -- 172.21.15.33:0/973277702 learned_addr learned my addr 172.21.15.33:0/973277702 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:16.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b64871640 1 -- 172.21.15.33:0/973277702 >> v1:172.21.15.138:6789/0 conn(0x7f5b6015e660 legacy=0x7f5b60071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:16.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b64871640 1 -- 172.21.15.33:0/973277702 >> v1:172.21.15.33:6789/0 conn(0x7f5b601444f0 legacy=0x7f5b600b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b5effd640 1 -- 172.21.15.33:0/973277702 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 981652753 0 0) 0x7f5b50001120 con 0x7f5b601444f0 2024-04-25T01:09:16.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b64871640 1 -- 172.21.15.33:0/973277702 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6015ef00 con 0x7f5b60072540 2024-04-25T01:09:16.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.652+0000 7f5b64871640 1 --2- 172.21.15.33:0/973277702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b601659f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009d90 tx=0x7f5b5801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1001df559605e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5effd640 1 -- 172.21.15.33:0/973277702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580b2070 con 0x7f5b60072540 2024-04-25T01:09:16.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5effd640 1 -- 172.21.15.33:0/973277702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b580a5760 con 0x7f5b60072540 2024-04-25T01:09:16.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5effd640 1 -- 172.21.15.33:0/973277702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b580a5b30 con 0x7f5b60072540 2024-04-25T01:09:16.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- 172.21.15.33:0/973277702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 msgr2=0x7f5b601659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 --2- 172.21.15.33:0/973277702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b601659f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009d90 tx=0x7f5b5801c8a0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- 172.21.15.33:0/973277702 shutdown_connections 2024-04-25T01:09:16.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 --2- 172.21.15.33:0/973277702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b601659f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- 172.21.15.33:0/973277702 >> 172.21.15.33:0/973277702 conn(0x7f5b600b1810 msgr2=0x7f5b600b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:16.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- 172.21.15.33:0/973277702 shutdown_connections 2024-04-25T01:09:16.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- 172.21.15.33:0/973277702 wait complete. 2024-04-25T01:09:16.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 Processor -- start 2024-04-25T01:09:16.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- start start 2024-04-25T01:09:16.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b60071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b601444f0 0x7f5b6013bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b6015e660 0x7f5b60070f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5b601688d0 con 0x7f5b60072540 2024-04-25T01:09:16.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b60168750 con 0x7f5b601444f0 2024-04-25T01:09:16.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5b60168a50 con 0x7f5b6015e660 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b60071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b60071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47020/0 (socket says 172.21.15.33:47020) 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 -- 172.21.15.33:0/201307036 learned_addr learned my addr 172.21.15.33:0/201307036 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b64871640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b6015e660 0x7f5b60070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5f7fe640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b601444f0 0x7f5b6013bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 -- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b6015e660 msgr2=0x7f5b60070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b6015e660 0x7f5b60070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 -- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b601444f0 msgr2=0x7f5b6013bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b601444f0 0x7f5b6013bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 -- 172.21.15.33:0/201307036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6013e030 con 0x7f5b60072540 2024-04-25T01:09:16.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b64871640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b6015e660 0x7f5b60070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5f7fe640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b601444f0 0x7f5b6013bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5ffff640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b60071d40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000c7b0 tx=0x7f5b5000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5cff9640 1 -- 172.21.15.33:0/201307036 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b50018070 con 0x7f5b60072540 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5cff9640 1 -- 172.21.15.33:0/201307036 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b500072f0 con 0x7f5b60072540 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b60072280 con 0x7f5b60072540 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b5cff9640 1 -- 172.21.15.33:0/201307036 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b500047f0 con 0x7f5b60072540 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.656+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6013bbe0 con 0x7f5b60072540 2024-04-25T01:09:16.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.660+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b60069040 con 0x7f5b60072540 2024-04-25T01:09:16.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.660+0000 7f5b5cff9640 1 -- 172.21.15.33:0/201307036 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5b5001b070 con 0x7f5b60072540 2024-04-25T01:09:16.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.660+0000 7f5b5cff9640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5b4c0678c0 0x7f5b4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.660+0000 7f5b5cff9640 1 -- 172.21.15.33:0/201307036 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5b50085720 con 0x7f5b60072540 2024-04-25T01:09:16.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.660+0000 7f5b5f7fe640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5b4c0678c0 0x7f5b4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.660+0000 7f5b5f7fe640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5b4c0678c0 0x7f5b4c069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b6006d6d0 tx=0x7f5b54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:16.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.664+0000 7f5b5cff9640 1 -- 172.21.15.33:0/201307036 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5b601448d0 con 0x7f5b60072540 2024-04-25T01:09:16.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.828+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5b6013dbe0 con 0x7f5b60072540 2024-04-25T01:09:16.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.828+0000 7f5b5cff9640 1 -- 172.21.15.33:0/201307036 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5b50055f40 con 0x7f5b60072540 2024-04-25T01:09:16.822 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5b4c0678c0 msgr2=0x7f5b4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5b4c0678c0 0x7f5b4c069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b6006d6d0 tx=0x7f5b54008040 comp rx=0 tx=0).stop 2024-04-25T01:09:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 msgr2=0x7f5b60071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b60071d40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5b5000c7b0 tx=0x7f5b5000cc70 comp rx=0 tx=0).stop 2024-04-25T01:09:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 shutdown_connections 2024-04-25T01:09:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5b4c0678c0 0x7f5b4c069d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5b6015e660 0x7f5b60070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b601444f0 0x7f5b6013bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 --2- 172.21.15.33:0/201307036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b60072540 0x7f5b60071d40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 >> 172.21.15.33:0/201307036 conn(0x7f5b600b1810 msgr2=0x7f5b6015dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 shutdown_connections 2024-04-25T01:09:16.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:16.832+0000 7f5b662d4640 1 -- 172.21.15.33:0/201307036 wait complete. 2024-04-25T01:09:16.839 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:09:16.839 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:16.846 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:16.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.000+0000 7f25b5687640 1 Processor -- start 2024-04-25T01:09:16.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25b5687640 1 -- start start 2024-04-25T01:09:16.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25b5687640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f25b0145470 0x7f25b015c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25b5687640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b00cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:16.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25b5687640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25b015f9a0 con 0x7f25b00cf0e0 2024-04-25T01:09:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25b5687640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f25b015f750 con 0x7f25b0145470 2024-04-25T01:09:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25b5687640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f25b015fc80 con 0x7f25b00b5530 2024-04-25T01:09:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25af7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b00cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25af7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b00cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:44842/0 (socket says 172.21.15.33:44842) 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25af7fe640 1 -- 172.21.15.33:0/4165132451 learned_addr learned my addr 172.21.15.33:0/4165132451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25aeffd640 1 --2- 172.21.15.33:0/4165132451 >> v2:172.21.15.138:3300/0 conn(0x7f25b0145470 0x7f25b015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25aeffd640 1 -- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 msgr2=0x7f25b00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25aeffd640 1 --2- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b00cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25aeffd640 1 -- 172.21.15.33:0/4165132451 >> v1:172.21.15.138:6789/0 conn(0x7f25b00cf0e0 legacy=0x7f25b00b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25aeffd640 1 -- 172.21.15.33:0/4165132451 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b015ff60 con 0x7f25b0145470 2024-04-25T01:09:16.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.004+0000 7f25af7fe640 1 --2- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b00cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:09:17.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25aeffd640 1 --2- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b015c160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25a00097c0 tx=0x7f25a001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5aa6c5df8df42f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25ae7fc640 1 -- 172.21.15.33:0/4165132451 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00b1070 con 0x7f25b0145470 2024-04-25T01:09:17.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25ae7fc640 1 -- 172.21.15.33:0/4165132451 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25a00ac410 con 0x7f25b0145470 2024-04-25T01:09:17.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25ae7fc640 1 -- 172.21.15.33:0/4165132451 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a00ac7e0 con 0x7f25b0145470 2024-04-25T01:09:17.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 -- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 msgr2=0x7f25b015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 --2- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b015c160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25a00097c0 tx=0x7f25a001c4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 -- 172.21.15.33:0/4165132451 shutdown_connections 2024-04-25T01:09:17.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 --2- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b00cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 --2- 172.21.15.33:0/4165132451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b015c160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 -- 172.21.15.33:0/4165132451 >> 172.21.15.33:0/4165132451 conn(0x7f25b00b1810 msgr2=0x7f25b00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 -- 172.21.15.33:0/4165132451 shutdown_connections 2024-04-25T01:09:17.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 -- 172.21.15.33:0/4165132451 wait complete. 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 Processor -- start 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.008+0000 7f25b5687640 1 -- start start 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b0144850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b00cf0e0 0x7f25b013a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b012c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0168c70 con 0x7f25b00cf0e0 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25b0168af0 con 0x7f25b0145470 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f25b0168df0 con 0x7f25b00b5530 2024-04-25T01:09:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b00cf0e0 0x7f25b013a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b00cf0e0 0x7f25b013a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47030/0 (socket says 172.21.15.33:47030) 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 -- 172.21.15.33:0/4255969927 learned_addr learned my addr 172.21.15.33:0/4255969927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25aeffd640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b0144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25af7fe640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b012c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 -- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 msgr2=0x7f25b0144850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b0144850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 -- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 msgr2=0x7f25b012c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b012c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 -- 172.21.15.33:0/4255969927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b01444e0 con 0x7f25b00cf0e0 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25aeffd640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b0144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25af7fe640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b012c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a67fc640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b00cf0e0 0x7f25b013a320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f259800c420 tx=0x7f259800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a7fff640 1 -- 172.21.15.33:0/4255969927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25980052d0 con 0x7f25b00cf0e0 2024-04-25T01:09:17.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a7fff640 1 -- 172.21.15.33:0/4255969927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2598005470 con 0x7f25b00cf0e0 2024-04-25T01:09:17.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b013a9b0 con 0x7f25b00cf0e0 2024-04-25T01:09:17.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a7fff640 1 -- 172.21.15.33:0/4255969927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2598007260 con 0x7f25b00cf0e0 2024-04-25T01:09:17.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b013af70 con 0x7f25b00cf0e0 2024-04-25T01:09:17.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25b013b630 con 0x7f25b00cf0e0 2024-04-25T01:09:17.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a7fff640 1 -- 172.21.15.33:0/4255969927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2598007400 con 0x7f25b00cf0e0 2024-04-25T01:09:17.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a7fff640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f25900678c0 0x7f2590069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25a7fff640 1 -- 172.21.15.33:0/4255969927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f25980855d0 con 0x7f25b00cf0e0 2024-04-25T01:09:17.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.012+0000 7f25aeffd640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f25900678c0 0x7f2590069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.016+0000 7f25aeffd640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f25900678c0 0x7f2590069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25a00ae000 tx=0x7f25a00a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.016+0000 7f25a7fff640 1 -- 172.21.15.33:0/4255969927 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f25b013b630 con 0x7f25b00cf0e0 2024-04-25T01:09:17.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.180+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25b0069040 con 0x7f25b00cf0e0 2024-04-25T01:09:17.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.180+0000 7f25a7fff640 1 -- 172.21.15.33:0/4255969927 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7f25b01444e0 con 0x7f25b00cf0e0 2024-04-25T01:09:17.176 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:17.176 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:17.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f25900678c0 msgr2=0x7f2590069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f25900678c0 0x7f2590069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f25a00ae000 tx=0x7f25a00a6040 comp rx=0 tx=0).stop 2024-04-25T01:09:17.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b00cf0e0 msgr2=0x7f25b013a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b00cf0e0 0x7f25b013a320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f259800c420 tx=0x7f259800c8e0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 shutdown_connections 2024-04-25T01:09:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f25900678c0 0x7f2590069d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b0145470 0x7f25b012c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b00cf0e0 0x7f25b013a320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 --2- 172.21.15.33:0/4255969927 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f25b00b5530 0x7f25b0144850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 >> 172.21.15.33:0/4255969927 conn(0x7f25b00b1810 msgr2=0x7f25b006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 shutdown_connections 2024-04-25T01:09:17.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.184+0000 7f25b5687640 1 -- 172.21.15.33:0/4255969927 wait complete. 2024-04-25T01:09:17.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:17.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.332+0000 7f1a540a8640 1 Processor -- start 2024-04-25T01:09:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.332+0000 7f1a540a8640 1 -- start start 2024-04-25T01:09:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a540a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d25e0 0x7f1a4c0cff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a540a8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a4c0d0490 0x7f1a4c167d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a540a8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a4c06f610 con 0x7f1a4c0cfb70 2024-04-25T01:09:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a540a8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1a4c0ce680 con 0x7f1a4c0d0490 2024-04-25T01:09:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a540a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c0ce960 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a4c0d0490 0x7f1a4c167d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a4c0d0490 0x7f1a4c167d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47046/0 (socket says 172.21.15.33:47046) 2024-04-25T01:09:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 -- 172.21.15.33:0/289046814 learned_addr learned my addr 172.21.15.33:0/289046814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a51643640 1 --2- 172.21.15.33:0/289046814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d25e0 0x7f1a4c0cff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 -- 172.21.15.33:0/289046814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d25e0 msgr2=0x7f1a4c0cff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 --2- 172.21.15.33:0/289046814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d25e0 0x7f1a4c0cff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 -- 172.21.15.33:0/289046814 >> v1:172.21.15.138:6789/0 conn(0x7f1a4c0cfb70 legacy=0x7f1a4c06ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 -- 172.21.15.33:0/289046814 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c0cec40 con 0x7f1a4c0d0490 2024-04-25T01:09:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.336+0000 7f1a52645640 1 --2- 172.21.15.33:0/289046814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d0490 0x7f1a4c167d00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009870 tx=0x7f1a3c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba693217370d4db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a50e42640 1 -- 172.21.15.33:0/289046814 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0b1070 con 0x7f1a4c0d0490 2024-04-25T01:09:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a50e42640 1 -- 172.21.15.33:0/289046814 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3c0ac620 con 0x7f1a4c0d0490 2024-04-25T01:09:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a50e42640 1 -- 172.21.15.33:0/289046814 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0ac9f0 con 0x7f1a4c0d0490 2024-04-25T01:09:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- 172.21.15.33:0/289046814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d0490 msgr2=0x7f1a4c167d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 --2- 172.21.15.33:0/289046814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d0490 0x7f1a4c167d00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009870 tx=0x7f1a3c01c4d0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- 172.21.15.33:0/289046814 shutdown_connections 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 --2- 172.21.15.33:0/289046814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d0490 0x7f1a4c167d00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 --2- 172.21.15.33:0/289046814 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d25e0 0x7f1a4c0cff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- 172.21.15.33:0/289046814 >> 172.21.15.33:0/289046814 conn(0x7f1a4c09a9d0 msgr2=0x7f1a4c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- 172.21.15.33:0/289046814 shutdown_connections 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- 172.21.15.33:0/289046814 wait complete. 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 Processor -- start 2024-04-25T01:09:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- start start 2024-04-25T01:09:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4c0cfb70 0x7f1a4c1586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d0490 0x7f1a4c160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d25e0 0x7f1a4c14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c16aa60 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4c070df0 con 0x7f1a4c0cfb70 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a540a8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a4c16abe0 con 0x7f1a4c0d0490 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d25e0 0x7f1a4c14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d25e0 0x7f1a4c14d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47062/0 (socket says 172.21.15.33:47062) 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 -- 172.21.15.33:0/2574937388 learned_addr learned my addr 172.21.15.33:0/2574937388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a51e44640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4c0cfb70 0x7f1a4c1586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 -- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d0490 msgr2=0x7f1a4c160620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d0490 0x7f1a4c160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 -- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4c0cfb70 msgr2=0x7f1a4c1586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4c0cfb70 0x7f1a4c1586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a52645640 1 -- 172.21.15.33:0/2574937388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4c0b0fc0 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.340+0000 7f1a51e44640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4c0cfb70 0x7f1a4c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a52645640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d25e0 0x7f1a4c14d3a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0af040 tx=0x7f1a3c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a4a7fc640 1 -- 172.21.15.33:0/2574937388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0b1040 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4c15ebb0 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4c161370 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a4a7fc640 1 -- 172.21.15.33:0/2574937388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a3c0ba070 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a4a7fc640 1 -- 172.21.15.33:0/2574937388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0bf660 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a2c0052a0 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a4a7fc640 1 -- 172.21.15.33:0/2574937388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1a3c0bf800 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a4a7fc640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a380678f0 0x7f1a38069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a4a7fc640 1 -- 172.21.15.33:0/2574937388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1a3c11fca0 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a51e44640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a380678f0 0x7f1a38069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.344+0000 7f1a51e44640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a380678f0 0x7f1a38069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a44001650 tx=0x7f1a44009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.348+0000 7f1a4a7fc640 1 -- 172.21.15.33:0/2574937388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1a4c0b0fc0 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a2c005860 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a4a7fc640 1 -- 172.21.15.33:0/2574937388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f1a3c016150 con 0x7f1a4c0d25e0 2024-04-25T01:09:17.505 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a380678f0 msgr2=0x7f1a38069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a380678f0 0x7f1a38069db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1a44001650 tx=0x7f1a44009040 comp rx=0 tx=0).stop 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d25e0 msgr2=0x7f1a4c14d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d25e0 0x7f1a4c14d3a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c0af040 tx=0x7f1a3c019270 comp rx=0 tx=0).stop 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 shutdown_connections 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a380678f0 0x7f1a38069db0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a4c0d25e0 0x7f1a4c14d3a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a4c0d0490 0x7f1a4c160620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 --2- 172.21.15.33:0/2574937388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4c0cfb70 0x7f1a4c1586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.512+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 >> 172.21.15.33:0/2574937388 conn(0x7f1a4c09a9d0 msgr2=0x7f1a4c06d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.516+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 shutdown_connections 2024-04-25T01:09:17.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.516+0000 7f1a540a8640 1 -- 172.21.15.33:0/2574937388 wait complete. 2024-04-25T01:09:17.525 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:17.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2143783640 1 Processor -- start 2024-04-25T01:09:17.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2143783640 1 -- start start 2024-04-25T01:09:17.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2143783640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f213c156600 0x7f213c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2143783640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f213c15cfb0 0x7f213c0d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2143783640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c070860 con 0x7f213c0d2140 2024-04-25T01:09:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2143783640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f213c0d25b0 con 0x7f213c15cfb0 2024-04-25T01:09:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2143783640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f213c06f440 con 0x7f213c156600 2024-04-25T01:09:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2140d1e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f213c15cfb0 0x7f213c0d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2141d20640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f213c0d2140 0x7f213c0cfb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:41204/0 (socket says 172.21.15.33:41204) 2024-04-25T01:09:17.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2141d20640 1 -- 172.21.15.33:0/23758013 learned_addr learned my addr 172.21.15.33:0/23758013 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f214151f640 1 --2- 172.21.15.33:0/23758013 >> v2:172.21.15.138:3300/0 conn(0x7f213c156600 0x7f213c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2140d1e640 1 -- 172.21.15.33:0/23758013 >> v2:172.21.15.138:3300/0 conn(0x7f213c156600 msgr2=0x7f213c06ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2140d1e640 1 --2- 172.21.15.33:0/23758013 >> v2:172.21.15.138:3300/0 conn(0x7f213c156600 0x7f213c06ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2140d1e640 1 -- 172.21.15.33:0/23758013 >> v1:172.21.15.33:6789/0 conn(0x7f213c0d2140 legacy=0x7f213c0cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f213bfff640 1 -- 172.21.15.33:0/23758013 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1966228179 0 0) 0x7f213c070860 con 0x7f213c0d2140 2024-04-25T01:09:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.656+0000 7f2140d1e640 1 -- 172.21.15.33:0/23758013 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c1569e0 con 0x7f213c15cfb0 2024-04-25T01:09:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2140d1e640 1 --2- 172.21.15.33:0/23758013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c15cfb0 0x7f213c0d1c00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f213c06ed00 tx=0x7f213001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8614836cc130d7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f213bfff640 1 -- 172.21.15.33:0/23758013 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21300b0070 con 0x7f213c15cfb0 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f213bfff640 1 -- 172.21.15.33:0/23758013 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21300ab600 con 0x7f213c15cfb0 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f213bfff640 1 -- 172.21.15.33:0/23758013 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21300ab9d0 con 0x7f213c15cfb0 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- 172.21.15.33:0/23758013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c15cfb0 msgr2=0x7f213c0d1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 --2- 172.21.15.33:0/23758013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c15cfb0 0x7f213c0d1c00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f213c06ed00 tx=0x7f213001c3f0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- 172.21.15.33:0/23758013 shutdown_connections 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 --2- 172.21.15.33:0/23758013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c15cfb0 0x7f213c0d1c00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 --2- 172.21.15.33:0/23758013 >> v2:172.21.15.138:3300/0 conn(0x7f213c156600 0x7f213c06ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- 172.21.15.33:0/23758013 >> 172.21.15.33:0/23758013 conn(0x7f213c09a9d0 msgr2=0x7f213c09ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- 172.21.15.33:0/23758013 shutdown_connections 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- 172.21.15.33:0/23758013 wait complete. 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 Processor -- start 2024-04-25T01:09:17.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- start start 2024-04-25T01:09:17.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c0d2140 0x7f213c1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213c156600 0x7f213c152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f213c15cfb0 0x7f213c15afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f213c0d2a60 con 0x7f213c0d2140 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f213c0d28e0 con 0x7f213c156600 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f2143783640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f213c0d0a90 con 0x7f213c15cfb0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.660+0000 7f214151f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c0d2140 0x7f213c1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c0d2140 0x7f213c1553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47076/0 (socket says 172.21.15.33:47076) 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 -- 172.21.15.33:0/2817441578 learned_addr learned my addr 172.21.15.33:0/2817441578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2141d20640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f213c15cfb0 0x7f213c15afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 -- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f213c15cfb0 msgr2=0x7f213c15afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2140d1e640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213c156600 0x7f213c152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f213c15cfb0 0x7f213c15afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 -- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213c156600 msgr2=0x7f213c152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213c156600 0x7f213c152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 -- 172.21.15.33:0/2817441578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c0b18f0 con 0x7f213c0d2140 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2141d20640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f213c15cfb0 0x7f213c15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f214151f640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c0d2140 0x7f213c1553e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f213400d800 tx=0x7f213400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2139ffb640 1 -- 172.21.15.33:0/2817441578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213400b6c0 con 0x7f213c0d2140 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2139ffb640 1 -- 172.21.15.33:0/2817441578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2134019070 con 0x7f213c0d2140 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f213c0b1bd0 con 0x7f213c0d2140 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2139ffb640 1 -- 172.21.15.33:0/2817441578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213400bae0 con 0x7f213c0d2140 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f213c0b2190 con 0x7f213c0d2140 2024-04-25T01:09:17.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f213c0b23d0 con 0x7f213c0d2140 2024-04-25T01:09:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2139ffb640 1 -- 172.21.15.33:0/2817441578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f213400bc80 con 0x7f213c0d2140 2024-04-25T01:09:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2139ffb640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f21280678c0 0x7f2128069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2139ffb640 1 -- 172.21.15.33:0/2817441578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2134086570 con 0x7f213c0d2140 2024-04-25T01:09:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2140d1e640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f21280678c0 0x7f2128069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.664+0000 7f2140d1e640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f21280678c0 0x7f2128069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21300ad000 tx=0x7f21300a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.668+0000 7f2139ffb640 1 -- 172.21.15.33:0/2817441578 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f213c0b23d0 con 0x7f213c0d2140 2024-04-25T01:09:17.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.824+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f213c069040 con 0x7f213c0d2140 2024-04-25T01:09:17.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.824+0000 7f2139ffb640 1 -- 172.21.15.33:0/2817441578 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f2134056d40 con 0x7f213c0d2140 2024-04-25T01:09:17.818 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f21280678c0 msgr2=0x7f2128069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f21280678c0 0x7f2128069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f21300ad000 tx=0x7f21300a6000 comp rx=0 tx=0).stop 2024-04-25T01:09:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c0d2140 msgr2=0x7f213c1553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c0d2140 0x7f213c1553e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f213400d800 tx=0x7f213400dcc0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 shutdown_connections 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f21280678c0 0x7f2128069d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f213c15cfb0 0x7f213c15afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f213c156600 0x7f213c152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 --2- 172.21.15.33:0/2817441578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f213c0d2140 0x7f213c1553e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 >> 172.21.15.33:0/2817441578 conn(0x7f213c09a9d0 msgr2=0x7f213c0aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 shutdown_connections 2024-04-25T01:09:17.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.828+0000 7f2143783640 1 -- 172.21.15.33:0/2817441578 wait complete. 2024-04-25T01:09:17.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:17.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 Processor -- start 2024-04-25T01:09:17.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 -- start start 2024-04-25T01:09:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7cc80b31a0 0x7f7cc80ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7cc80aec20 0x7f7cc8161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc815e480 con 0x7f7cc80b31a0 2024-04-25T01:09:17.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f7cc806df00 con 0x7f7cc80aec20 2024-04-25T01:09:17.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7ccd147640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc80ae220 con 0x7f7cc815e0a0 2024-04-25T01:09:17.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7cc80b31a0 0x7f7cc80ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7cc80b31a0 0x7f7cc80ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47086/0 (socket says 172.21.15.33:47086) 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 -- 172.21.15.33:0/3526352977 learned_addr learned my addr 172.21.15.33:0/3526352977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc6d9d640 1 --2- 172.21.15.33:0/3526352977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc759e640 1 --2- 172.21.15.33:0/3526352977 >> v2:172.21.15.138:3300/0 conn(0x7f7cc80aec20 0x7f7cc8161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 -- 172.21.15.33:0/3526352977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 msgr2=0x7f7cc80716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 --2- 172.21.15.33:0/3526352977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 -- 172.21.15.33:0/3526352977 >> v2:172.21.15.138:3300/0 conn(0x7f7cc80aec20 msgr2=0x7f7cc8161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:17.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 --2- 172.21.15.33:0/3526352977 >> v2:172.21.15.138:3300/0 conn(0x7f7cc80aec20 0x7f7cc8161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.956+0000 7f7cc659c640 1 -- 172.21.15.33:0/3526352977 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc806e880 con 0x7f7cc80b31a0 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7cc659c640 1 --2- 172.21.15.33:0/3526352977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80b31a0 0x7f7cc80ae6e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80b17a0 tx=0x7f7cbc01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ebc80a75c21d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7cc5d9b640 1 -- 172.21.15.33:0/3526352977 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0b0070 con 0x7f7cc80b31a0 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7cc5d9b640 1 -- 172.21.15.33:0/3526352977 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cbc0ab600 con 0x7f7cc80b31a0 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7cc5d9b640 1 -- 172.21.15.33:0/3526352977 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc0ab9d0 con 0x7f7cc80b31a0 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 -- 172.21.15.33:0/3526352977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80b31a0 msgr2=0x7f7cc80ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 --2- 172.21.15.33:0/3526352977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80b31a0 0x7f7cc80ae6e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7cc80b17a0 tx=0x7f7cbc01c1d0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 -- 172.21.15.33:0/3526352977 shutdown_connections 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 --2- 172.21.15.33:0/3526352977 >> v2:172.21.15.138:3300/0 conn(0x7f7cc80aec20 0x7f7cc8161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 --2- 172.21.15.33:0/3526352977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80b31a0 0x7f7cc80ae6e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 --2- 172.21.15.33:0/3526352977 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 -- 172.21.15.33:0/3526352977 >> 172.21.15.33:0/3526352977 conn(0x7f7cc80b1810 msgr2=0x7f7cc80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 -- 172.21.15.33:0/3526352977 shutdown_connections 2024-04-25T01:09:17.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 -- 172.21.15.33:0/3526352977 wait complete. 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 Processor -- start 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.960+0000 7f7ccd147640 1 -- start start 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80aec20 0x7f7cc81552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cc80b31a0 0x7f7cc814d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80af030 con 0x7f7cc80aec20 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80b3810 con 0x7f7cc80b31a0 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc80af1b0 con 0x7f7cc815e0a0 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80aec20 0x7f7cc81552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80aec20 0x7f7cc81552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47098/0 (socket says 172.21.15.33:47098) 2024-04-25T01:09:17.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 -- 172.21.15.33:0/2076827248 learned_addr learned my addr 172.21.15.33:0/2076827248 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc759e640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc659c640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cc80b31a0 0x7f7cc814d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 -- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 msgr2=0x7f7cc80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 -- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cc80b31a0 msgr2=0x7f7cc814d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cc80b31a0 0x7f7cc814d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 -- 172.21.15.33:0/2076827248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc814ded0 con 0x7f7cc80aec20 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc659c640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cc80b31a0 0x7f7cc814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc6d9d640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80aec20 0x7f7cc81552a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800d5f0 tx=0x7f7cb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7caf7fe640 1 -- 172.21.15.33:0/2076827248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80125f0 con 0x7f7cc80aec20 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7caf7fe640 1 -- 172.21.15.33:0/2076827248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cb8019070 con 0x7f7cc80aec20 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc80ccab0 con 0x7f7cc80aec20 2024-04-25T01:09:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7caf7fe640 1 -- 172.21.15.33:0/2076827248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80129d0 con 0x7f7cc80aec20 2024-04-25T01:09:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc80cd070 con 0x7f7cc80aec20 2024-04-25T01:09:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc814e0a0 con 0x7f7cc80aec20 2024-04-25T01:09:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7caf7fe640 1 -- 172.21.15.33:0/2076827248 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7cb8012b70 con 0x7f7cc80aec20 2024-04-25T01:09:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7caf7fe640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7cb40678f0 0x7f7cb4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7caf7fe640 1 -- 172.21.15.33:0/2076827248 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7cb8086520 con 0x7f7cc80aec20 2024-04-25T01:09:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.964+0000 7f7cc659c640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7cb40678f0 0x7f7cb4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.968+0000 7f7cc659c640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7cb40678f0 0x7f7cb4069db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0ad000 tx=0x7f7cbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:17.968+0000 7f7caf7fe640 1 -- 172.21.15.33:0/2076827248 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7cb8056c60 con 0x7f7cc80aec20 2024-04-25T01:09:18.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7cc8069040 con 0x7f7cc80aec20 2024-04-25T01:09:18.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7caf7fe640 1 -- 172.21.15.33:0/2076827248 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f7cb8056560 con 0x7f7cc80aec20 2024-04-25T01:09:18.117 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:18.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7cb40678f0 msgr2=0x7f7cb4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7cb40678f0 0x7f7cb4069db0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0ad000 tx=0x7f7cbc008040 comp rx=0 tx=0).stop 2024-04-25T01:09:18.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80aec20 msgr2=0x7f7cc81552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80aec20 0x7f7cc81552a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800d5f0 tx=0x7f7cb800dab0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 shutdown_connections 2024-04-25T01:09:18.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7cb40678f0 0x7f7cb4069db0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7cc815e0a0 0x7f7cc80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cc80b31a0 0x7f7cc814d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 --2- 172.21.15.33:0/2076827248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cc80aec20 0x7f7cc81552a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 >> 172.21.15.33:0/2076827248 conn(0x7f7cc80b1810 msgr2=0x7f7cc80afb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:18.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 shutdown_connections 2024-04-25T01:09:18.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.124+0000 7f7ccd147640 1 -- 172.21.15.33:0/2076827248 wait complete. 2024-04-25T01:09:18.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac59564640 1 Processor -- start 2024-04-25T01:09:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac59564640 1 -- start start 2024-04-25T01:09:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac59564640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac5415e0d0 0x7fac540cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac59564640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d2ac0 0x7fac54167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac59564640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac540b6cc0 con 0x7fac540d1f60 2024-04-25T01:09:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac59564640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fac540b7580 con 0x7fac5415e0d0 2024-04-25T01:09:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac59564640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac540b6f30 con 0x7fac540d2ac0 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac52ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac5415e0d0 0x7fac540cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac537fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d2ac0 0x7fac54167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac537fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d2ac0 0x7fac54167cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:44908/0 (socket says 172.21.15.33:44908) 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac52ffd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac5415e0d0 0x7fac540cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:60842/0 (socket says 172.21.15.33:60842) 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac537fe640 1 -- 172.21.15.33:0/1619238557 learned_addr learned my addr 172.21.15.33:0/1619238557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac52ffd640 1 -- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d2ac0 msgr2=0x7fac54167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac52ffd640 1 --2- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d2ac0 0x7fac54167cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac52ffd640 1 -- 172.21.15.33:0/1619238557 >> v1:172.21.15.138:6789/0 conn(0x7fac540d1f60 legacy=0x7fac540d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac52ffd640 1 -- 172.21.15.33:0/1619238557 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac540b7210 con 0x7fac5415e0d0 2024-04-25T01:09:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.268+0000 7fac537fe640 1 --2- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d2ac0 0x7fac54167cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:09:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac52ffd640 1 --2- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac5415e0d0 0x7fac540cf3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac44009d90 tx=0x7fac4401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce32ed97aa5263e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac51ffb640 1 -- 172.21.15.33:0/1619238557 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac440b2070 con 0x7fac5415e0d0 2024-04-25T01:09:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac51ffb640 1 -- 172.21.15.33:0/1619238557 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac440a5960 con 0x7fac5415e0d0 2024-04-25T01:09:18.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac51ffb640 1 -- 172.21.15.33:0/1619238557 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac440a5d30 con 0x7fac5415e0d0 2024-04-25T01:09:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 -- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac5415e0d0 msgr2=0x7fac540cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 --2- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac5415e0d0 0x7fac540cf3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fac44009d90 tx=0x7fac4401c970 comp rx=0 tx=0).stop 2024-04-25T01:09:18.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 -- 172.21.15.33:0/1619238557 shutdown_connections 2024-04-25T01:09:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 --2- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d2ac0 0x7fac54167cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 --2- 172.21.15.33:0/1619238557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac5415e0d0 0x7fac540cf3f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 -- 172.21.15.33:0/1619238557 >> 172.21.15.33:0/1619238557 conn(0x7fac540d3dd0 msgr2=0x7fac540d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 -- 172.21.15.33:0/1619238557 shutdown_connections 2024-04-25T01:09:18.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.272+0000 7fac59564640 1 -- 172.21.15.33:0/1619238557 wait complete. 2024-04-25T01:09:18.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 Processor -- start 2024-04-25T01:09:18.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 -- start start 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d1f60 0x7fac54154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac540d2ac0 0x7fac541fb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5415e0d0 0x7fac5415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac5416abd0 con 0x7fac5415e0d0 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac5416aa50 con 0x7fac540d2ac0 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac5416ad50 con 0x7fac540d1f60 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5415e0d0 0x7fac5415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5415e0d0 0x7fac5415b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47100/0 (socket says 172.21.15.33:47100) 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 -- 172.21.15.33:0/35546136 learned_addr learned my addr 172.21.15.33:0/35546136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac52ffd640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d1f60 0x7fac54154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac527fc640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac540d2ac0 0x7fac541fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 -- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d1f60 msgr2=0x7fac54154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d1f60 0x7fac54154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 -- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac540d2ac0 msgr2=0x7fac541fb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac540d2ac0 0x7fac541fb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 -- 172.21.15.33:0/35546136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac541fb5d0 con 0x7fac5415e0d0 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac537fe640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5415e0d0 0x7fac5415b9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fac4c006e10 tx=0x7fac4c009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac3b7fe640 1 -- 172.21.15.33:0/35546136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00ca00 con 0x7fac5415e0d0 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac3b7fe640 1 -- 172.21.15.33:0/35546136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac4c00cba0 con 0x7fac5415e0d0 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5415fe60 con 0x7fac5415e0d0 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac3b7fe640 1 -- 172.21.15.33:0/35546136 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c004030 con 0x7fac5415e0d0 2024-04-25T01:09:18.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac54160420 con 0x7fac5415e0d0 2024-04-25T01:09:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.276+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac54069040 con 0x7fac5415e0d0 2024-04-25T01:09:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.280+0000 7fac3b7fe640 1 -- 172.21.15.33:0/35546136 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac4c0041d0 con 0x7fac5415e0d0 2024-04-25T01:09:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.280+0000 7fac3b7fe640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac400678f0 0x7fac40069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.280+0000 7fac3b7fe640 1 -- 172.21.15.33:0/35546136 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fac4c086780 con 0x7fac5415e0d0 2024-04-25T01:09:18.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.280+0000 7fac52ffd640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac400678f0 0x7fac40069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.280+0000 7fac52ffd640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac400678f0 0x7fac40069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac440af000 tx=0x7fac440a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:18.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.284+0000 7fac3b7fe640 1 -- 172.21.15.33:0/35546136 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fac541fb5d0 con 0x7fac5415e0d0 2024-04-25T01:09:18.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fac540cf5d0 con 0x7fac5415e0d0 2024-04-25T01:09:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac3b7fe640 1 -- 172.21.15.33:0/35546136 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fac4c017020 con 0x7fac5415e0d0 2024-04-25T01:09:18.432 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac400678f0 msgr2=0x7fac40069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac400678f0 0x7fac40069db0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fac440af000 tx=0x7fac440a7070 comp rx=0 tx=0).stop 2024-04-25T01:09:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5415e0d0 msgr2=0x7fac5415b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5415e0d0 0x7fac5415b9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fac4c006e10 tx=0x7fac4c009e00 comp rx=0 tx=0).stop 2024-04-25T01:09:18.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 shutdown_connections 2024-04-25T01:09:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac400678f0 0x7fac40069db0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5415e0d0 0x7fac5415b9c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac540d2ac0 0x7fac541fb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 --2- 172.21.15.33:0/35546136 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac540d1f60 0x7fac54154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 >> 172.21.15.33:0/35546136 conn(0x7fac540d3dd0 msgr2=0x7fac540d1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 shutdown_connections 2024-04-25T01:09:18.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.440+0000 7fac59564640 1 -- 172.21.15.33:0/35546136 wait complete. 2024-04-25T01:09:18.451 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81eb38640 1 Processor -- start 2024-04-25T01:09:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81eb38640 1 -- start start 2024-04-25T01:09:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81eb38640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe818072500 0x7fe8180b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81eb38640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8180b4060 0x7fe81806e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81eb38640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8180b4850 con 0x7fe81815b060 2024-04-25T01:09:18.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81eb38640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fe8180b4b30 con 0x7fe8180b4060 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81eb38640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fe81806de90 con 0x7fe818072500 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81d0d5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8180b4060 0x7fe81806e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81c8d4640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe81815b060 0x7fe818071df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49022/0 (socket says 172.21.15.33:49022) 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81c8d4640 1 -- 172.21.15.33:0/545511236 learned_addr learned my addr 172.21.15.33:0/545511236 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81d0d5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8180b4060 0x7fe81806e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60014/0 (socket says 172.21.15.33:60014) 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81d0d5640 1 -- 172.21.15.33:0/545511236 >> v2:172.21.15.138:3300/0 conn(0x7fe818072500 msgr2=0x7fe8180b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81d0d5640 1 --2- 172.21.15.33:0/545511236 >> v2:172.21.15.138:3300/0 conn(0x7fe818072500 0x7fe8180b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81d0d5640 1 -- 172.21.15.33:0/545511236 >> v1:172.21.15.33:6789/0 conn(0x7fe81815b060 legacy=0x7fe818071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.572+0000 7fe81d0d5640 1 -- 172.21.15.33:0/545511236 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81806e170 con 0x7fe8180b4060 2024-04-25T01:09:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 --2- 172.21.15.33:0/545511236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180b4060 0x7fe81806e970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe818070780 tx=0x7fe81001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5f5ca5b5fc24e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe8177fe640 1 -- 172.21.15.33:0/545511236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81000b2d0 con 0x7fe8180b4060 2024-04-25T01:09:18.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe8177fe640 1 -- 172.21.15.33:0/545511236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8100a7910 con 0x7fe8180b4060 2024-04-25T01:09:18.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe8177fe640 1 -- 172.21.15.33:0/545511236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100a7ce0 con 0x7fe8180b4060 2024-04-25T01:09:18.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- 172.21.15.33:0/545511236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180b4060 msgr2=0x7fe81806e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 --2- 172.21.15.33:0/545511236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180b4060 0x7fe81806e970 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe818070780 tx=0x7fe81001eb30 comp rx=0 tx=0).stop 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- 172.21.15.33:0/545511236 shutdown_connections 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 --2- 172.21.15.33:0/545511236 >> v2:172.21.15.138:3300/0 conn(0x7fe818072500 0x7fe8180b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 --2- 172.21.15.33:0/545511236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8180b4060 0x7fe81806e970 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- 172.21.15.33:0/545511236 >> 172.21.15.33:0/545511236 conn(0x7fe8180b27b0 msgr2=0x7fe8180b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- 172.21.15.33:0/545511236 shutdown_connections 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- 172.21.15.33:0/545511236 wait complete. 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 Processor -- start 2024-04-25T01:09:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- start start 2024-04-25T01:09:18.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe818072500 0x7fe81814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8180b4060 0x7fe818149bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe81815b060 0x7fe8180cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe81806f840 con 0x7fe81815b060 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe81806f6c0 con 0x7fe818072500 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe81806f9c0 con 0x7fe8180b4060 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe81815b060 0x7fe8180cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe81815b060 0x7fe8180cf610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60028/0 (socket says 172.21.15.33:60028) 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 -- 172.21.15.33:0/1832545507 learned_addr learned my addr 172.21.15.33:0/1832545507 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81c8d4640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe818072500 0x7fe81814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 -- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8180b4060 msgr2=0x7fe818149bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe817fff640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8180b4060 0x7fe818149bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8180b4060 0x7fe818149bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 -- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe818072500 msgr2=0x7fe81814c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe818072500 0x7fe81814c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 -- 172.21.15.33:0/1832545507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe818155680 con 0x7fe81815b060 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81c8d4640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe818072500 0x7fe81814c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81d0d5640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe81815b060 0x7fe8180cf610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe81001c760 tx=0x7fe81001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe8157fa640 1 -- 172.21.15.33:0/1832545507 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100be020 con 0x7fe81815b060 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe8157fa640 1 -- 172.21.15.33:0/1832545507 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe810009950 con 0x7fe81815b060 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe818155960 con 0x7fe81815b060 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe8157fa640 1 -- 172.21.15.33:0/1832545507 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810009d40 con 0x7fe81815b060 2024-04-25T01:09:18.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.576+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe818156440 con 0x7fe81815b060 2024-04-25T01:09:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.580+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe818069040 con 0x7fe81815b060 2024-04-25T01:09:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.580+0000 7fe8157fa640 1 -- 172.21.15.33:0/1832545507 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe8100c1020 con 0x7fe81815b060 2024-04-25T01:09:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.580+0000 7fe8157fa640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe7fc0678c0 0x7fe7fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.580+0000 7fe8157fa640 1 -- 172.21.15.33:0/1832545507 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe810123540 con 0x7fe81815b060 2024-04-25T01:09:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.580+0000 7fe81c8d4640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe7fc0678c0 0x7fe7fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:18.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.580+0000 7fe81c8d4640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe7fc0678c0 0x7fe7fc069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe804007fd0 tx=0x7fe804009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:18.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.584+0000 7fe8157fa640 1 -- 172.21.15.33:0/1832545507 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe8100c8030 con 0x7fe81815b060 2024-04-25T01:09:18.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.756+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe8180b4e00 con 0x7fe81815b060 2024-04-25T01:09:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.756+0000 7fe8157fa640 1 -- 172.21.15.33:0/1832545507 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fe810018720 con 0x7fe81815b060 2024-04-25T01:09:18.751 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:18.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe7fc0678c0 msgr2=0x7fe7fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe7fc0678c0 0x7fe7fc069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe804007fd0 tx=0x7fe804009040 comp rx=0 tx=0).stop 2024-04-25T01:09:18.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe81815b060 msgr2=0x7fe8180cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe81815b060 0x7fe8180cf610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe81001c760 tx=0x7fe81001b8d0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 shutdown_connections 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe7fc0678c0 0x7fe7fc069d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe81815b060 0x7fe8180cf610 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8180b4060 0x7fe818149bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 --2- 172.21.15.33:0/1832545507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe818072500 0x7fe81814c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 >> 172.21.15.33:0/1832545507 conn(0x7fe8180b27b0 msgr2=0x7fe818150c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 shutdown_connections 2024-04-25T01:09:18.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:18.760+0000 7fe81eb38640 1 -- 172.21.15.33:0/1832545507 wait complete. 2024-04-25T01:09:18.771 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-04-25T01:09:18.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:18.771 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:18.784 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:18.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:09:18.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:18.785 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T01:09:18.832 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:09:18.832 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:18.838 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:09:18.839 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:18.887 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:19.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.020+0000 7f2ac0bd5640 1 Processor -- start 2024-04-25T01:09:19.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.020+0000 7f2ac0bd5640 1 -- start start 2024-04-25T01:09:19.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2abc1444f0 0x7f2abc0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2abc15e660 0x7f2abc071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2abc1448d0 con 0x7f2abc072370 2024-04-25T01:09:19.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f2abc0727e0 con 0x7f2abc1444f0 2024-04-25T01:09:19.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2abc0ce0c0 con 0x7f2abc15e660 2024-04-25T01:09:19.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2abc1444f0 0x7f2abc0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2abc1444f0 0x7f2abc0b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60044/0 (socket says 172.21.15.33:60044) 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 -- 172.21.15.33:0/1845043511 learned_addr learned my addr 172.21.15.33:0/1845043511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ab9d9b640 1 --2- 172.21.15.33:0/1845043511 >> v2:172.21.15.138:3300/0 conn(0x7f2abc15e660 0x7f2abc071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 -- 172.21.15.33:0/1845043511 >> v2:172.21.15.138:3300/0 conn(0x7f2abc15e660 msgr2=0x7f2abc071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 --2- 172.21.15.33:0/1845043511 >> v2:172.21.15.138:3300/0 conn(0x7f2abc15e660 0x7f2abc071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 -- 172.21.15.33:0/1845043511 >> v1:172.21.15.33:6789/0 conn(0x7f2abc072370 legacy=0x7f2abc06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 -- 172.21.15.33:0/1845043511 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc0b5b60 con 0x7f2abc1444f0 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ab959a640 1 -- 172.21.15.33:0/1845043511 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 963378126 0 0) 0x7f2ab40014a0 con 0x7f2abc072370 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ab9d9b640 1 --2- 172.21.15.33:0/1845043511 >> v2:172.21.15.138:3300/0 conn(0x7f2abc15e660 0x7f2abc071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2aba59c640 1 --2- 172.21.15.33:0/1845043511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc0b5620 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0b5dd0 tx=0x7f2aac01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7ca9a7bcda6069 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ab959a640 1 -- 172.21.15.33:0/1845043511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0b4070 con 0x7f2abc1444f0 2024-04-25T01:09:19.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ab959a640 1 -- 172.21.15.33:0/1845043511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aac0a7950 con 0x7f2abc1444f0 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ab959a640 1 -- 172.21.15.33:0/1845043511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0a7d20 con 0x7f2abc1444f0 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/1845043511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 msgr2=0x7f2abc0b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/1845043511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc0b5620 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0b5dd0 tx=0x7f2aac01eb60 comp rx=0 tx=0).stop 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/1845043511 shutdown_connections 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/1845043511 >> v2:172.21.15.138:3300/0 conn(0x7f2abc15e660 0x7f2abc071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/1845043511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc0b5620 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/1845043511 >> 172.21.15.33:0/1845043511 conn(0x7f2abc0b1810 msgr2=0x7f2abc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/1845043511 shutdown_connections 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.024+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/1845043511 wait complete. 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 Processor -- start 2024-04-25T01:09:19.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 -- start start 2024-04-25T01:09:19.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2abc072370 0x7f2abc1f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc1f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2abc15e660 0x7f2abc1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc0b3620 con 0x7f2abc1444f0 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2abc15f4c0 con 0x7f2abc072370 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2abc0b37a0 con 0x7f2abc15e660 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc1f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc1f83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60056/0 (socket says 172.21.15.33:60056) 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 -- 172.21.15.33:0/652347729 learned_addr learned my addr 172.21.15.33:0/652347729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 -- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2abc15e660 msgr2=0x7f2abc1645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2abc15e660 0x7f2abc1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 -- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2abc072370 msgr2=0x7f2abc1f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2abc072370 0x7f2abc1f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 -- 172.21.15.33:0/652347729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2abc1f7880 con 0x7f2abc1444f0 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ab9d9b640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc1f83d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000c7f0 tx=0x7f2ab000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2aaaffd640 1 -- 172.21.15.33:0/652347729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0015070 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2aaaffd640 1 -- 172.21.15.33:0/652347729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab0007820 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2aaaffd640 1 -- 172.21.15.33:0/652347729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0007c10 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2abc1f7b60 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2abc131270 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.028+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2abc131470 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.032+0000 7f2aaaffd640 1 -- 172.21.15.33:0/652347729 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ab0018070 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.032+0000 7f2aaaffd640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a9c0678c0 0x7f2a9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.032+0000 7f2aaaffd640 1 -- 172.21.15.33:0/652347729 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2ab00862d0 con 0x7f2abc1444f0 2024-04-25T01:09:19.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.032+0000 7f2aba59c640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a9c0678c0 0x7f2a9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.032+0000 7f2aba59c640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a9c0678c0 0x7f2a9c069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0b1000 tx=0x7f2aac004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.036+0000 7f2aaaffd640 1 -- 172.21.15.33:0/652347729 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2abc131470 con 0x7f2abc1444f0 2024-04-25T01:09:19.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.196+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2abc069040 con 0x7f2abc1444f0 2024-04-25T01:09:19.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2aaaffd640 1 -- 172.21.15.33:0/652347729 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7f2ab0056970 con 0x7f2abc1444f0 2024-04-25T01:09:19.193 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:19.193 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a9c0678c0 msgr2=0x7f2a9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a9c0678c0 0x7f2a9c069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2aac0b1000 tx=0x7f2aac004740 comp rx=0 tx=0).stop 2024-04-25T01:09:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 msgr2=0x7f2abc1f83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc1f83d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000c7f0 tx=0x7f2ab000ccb0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 shutdown_connections 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2a9c0678c0 0x7f2a9c069d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2abc15e660 0x7f2abc1645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2abc1444f0 0x7f2abc1f83d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 --2- 172.21.15.33:0/652347729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2abc072370 0x7f2abc1f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 >> 172.21.15.33:0/652347729 conn(0x7f2abc0b1810 msgr2=0x7f2abc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 shutdown_connections 2024-04-25T01:09:19.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.200+0000 7f2ac0bd5640 1 -- 172.21.15.33:0/652347729 wait complete. 2024-04-25T01:09:19.211 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:19.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb22d2e640 1 Processor -- start 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb22d2e640 1 -- start start 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb22d2e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb1c06ecc0 0x7fcb1c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb22d2e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb1c0b4a20 con 0x7fcb1c15ae20 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb22d2e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb1c0b2bb0 con 0x7fcb1c072410 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb22d2e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fcb1c0b2e90 con 0x7fcb1c06ecc0 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb1bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb1c06ecc0 0x7fcb1c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb20aca640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcb1c15ae20 0x7fcb1c06e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49044/0 (socket says 172.21.15.33:49044) 2024-04-25T01:09:19.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb20aca640 1 -- 172.21.15.33:0/1937743505 learned_addr learned my addr 172.21.15.33:0/1937743505 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:19.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb1bfff640 1 -- 172.21.15.33:0/1937743505 >> v1:172.21.15.138:6789/0 conn(0x7fcb1c072410 legacy=0x7fcb1c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:19.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb1bfff640 1 -- 172.21.15.33:0/1937743505 >> v1:172.21.15.33:6789/0 conn(0x7fcb1c15ae20 legacy=0x7fcb1c06e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb1bfff640 1 -- 172.21.15.33:0/1937743505 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c0b3170 con 0x7fcb1c06ecc0 2024-04-25T01:09:19.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.336+0000 7fcb1bfff640 1 --2- 172.21.15.33:0/1937743505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c06ecc0 0x7fcb1c0b3690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb100097c0 tx=0x7fcb1001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64728d57a7acc8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb1b7fe640 1 -- 172.21.15.33:0/1937743505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100ab660 con 0x7fcb1c06ecc0 2024-04-25T01:09:19.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb1b7fe640 1 -- 172.21.15.33:0/1937743505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb100b0070 con 0x7fcb1c06ecc0 2024-04-25T01:09:19.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb1b7fe640 1 -- 172.21.15.33:0/1937743505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100aba20 con 0x7fcb1c06ecc0 2024-04-25T01:09:19.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- 172.21.15.33:0/1937743505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c06ecc0 msgr2=0x7fcb1c0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/1937743505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c06ecc0 0x7fcb1c0b3690 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcb100097c0 tx=0x7fcb1001c460 comp rx=0 tx=0).stop 2024-04-25T01:09:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- 172.21.15.33:0/1937743505 shutdown_connections 2024-04-25T01:09:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/1937743505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c06ecc0 0x7fcb1c0b3690 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- 172.21.15.33:0/1937743505 >> 172.21.15.33:0/1937743505 conn(0x7fcb1c0b1810 msgr2=0x7fcb1c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- 172.21.15.33:0/1937743505 shutdown_connections 2024-04-25T01:09:19.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- 172.21.15.33:0/1937743505 wait complete. 2024-04-25T01:09:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 Processor -- start 2024-04-25T01:09:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- start start 2024-04-25T01:09:19.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb1c06ecc0 0x7fcb1c1490e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c072410 0x7fcb1c14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c15ae20 0x7fcb1c156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c168d90 con 0x7fcb1c15ae20 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1c168c10 con 0x7fcb1c072410 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcb1c168f10 con 0x7fcb1c06ecc0 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c15ae20 0x7fcb1c156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c15ae20 0x7fcb1c156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60068/0 (socket says 172.21.15.33:60068) 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 -- 172.21.15.33:0/343687614 learned_addr learned my addr 172.21.15.33:0/343687614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb1bfff640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c072410 0x7fcb1c14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb20aca640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb1c06ecc0 0x7fcb1c1490e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 -- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb1c06ecc0 msgr2=0x7fcb1c1490e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb1c06ecc0 0x7fcb1c1490e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 -- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c072410 msgr2=0x7fcb1c14e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c072410 0x7fcb1c14e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 -- 172.21.15.33:0/343687614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1c157290 con 0x7fcb1c15ae20 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb212cb640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c15ae20 0x7fcb1c156d50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c410 tx=0x7fcb0c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb197fa640 1 -- 172.21.15.33:0/343687614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c017070 con 0x7fcb1c15ae20 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb197fa640 1 -- 172.21.15.33:0/343687614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb0c003600 con 0x7fcb1c15ae20 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb197fa640 1 -- 172.21.15.33:0/343687614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c003a30 con 0x7fcb1c15ae20 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1c157560 con 0x7fcb1c15ae20 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.340+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1c14f020 con 0x7fcb1c15ae20 2024-04-25T01:09:19.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.344+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb1c069040 con 0x7fcb1c15ae20 2024-04-25T01:09:19.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.344+0000 7fcb197fa640 1 -- 172.21.15.33:0/343687614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb0c01a070 con 0x7fcb1c15ae20 2024-04-25T01:09:19.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.344+0000 7fcb197fa640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb080678f0 0x7fcb08069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.344+0000 7fcb197fa640 1 -- 172.21.15.33:0/343687614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcb0c0853f0 con 0x7fcb1c15ae20 2024-04-25T01:09:19.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.344+0000 7fcb20aca640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb080678f0 0x7fcb08069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.344+0000 7fcb20aca640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb080678f0 0x7fcb08069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c151690 tx=0x7fcb14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.348+0000 7fcb197fa640 1 -- 172.21.15.33:0/343687614 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb0c055bc0 con 0x7fcb1c15ae20 2024-04-25T01:09:19.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.508+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcb1c06ce50 con 0x7fcb1c15ae20 2024-04-25T01:09:19.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb197fa640 1 -- 172.21.15.33:0/343687614 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fcb0c0554c0 con 0x7fcb1c15ae20 2024-04-25T01:09:19.504 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb080678f0 msgr2=0x7fcb08069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb080678f0 0x7fcb08069db0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c151690 tx=0x7fcb14009040 comp rx=0 tx=0).stop 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c15ae20 msgr2=0x7fcb1c156d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c15ae20 0x7fcb1c156d50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c00c410 tx=0x7fcb0c00c8d0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 shutdown_connections 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb080678f0 0x7fcb08069db0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb1c15ae20 0x7fcb1c156d50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb1c072410 0x7fcb1c14e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 --2- 172.21.15.33:0/343687614 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb1c06ecc0 0x7fcb1c1490e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 >> 172.21.15.33:0/343687614 conn(0x7fcb1c0b1810 msgr2=0x7fcb1c0ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 shutdown_connections 2024-04-25T01:09:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.512+0000 7fcb22d2e640 1 -- 172.21.15.33:0/343687614 wait complete. 2024-04-25T01:09:19.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f0061f29640 1 Processor -- start 2024-04-25T01:09:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f0061f29640 1 -- start start 2024-04-25T01:09:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f0061f29640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c15f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f0061f29640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f005c15b110 0x7f005c071e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f0061f29640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f005c15b940 con 0x7f005c072390 2024-04-25T01:09:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f0061f29640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f005c06de90 con 0x7f005c15b110 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f0061f29640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f005c06e170 con 0x7f005c158910 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f005affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f005c15b110 0x7f005c071e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f005affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f005c15b110 0x7f005c071e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60080/0 (socket says 172.21.15.33:60080) 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.656+0000 7f005affd640 1 -- 172.21.15.33:0/2932722265 learned_addr learned my addr 172.21.15.33:0/2932722265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005b7fe640 1 --2- 172.21.15.33:0/2932722265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005affd640 1 -- 172.21.15.33:0/2932722265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 msgr2=0x7f005c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005affd640 1 --2- 172.21.15.33:0/2932722265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c15f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005affd640 1 -- 172.21.15.33:0/2932722265 >> v1:172.21.15.138:6789/0 conn(0x7f005c072390 legacy=0x7f005c0ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005affd640 1 -- 172.21.15.33:0/2932722265 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c15bc20 con 0x7f005c15b110 2024-04-25T01:09:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005affd640 1 --2- 172.21.15.33:0/2932722265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c071e50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0050009870 tx=0x7f005001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b69bfe5a9eb51a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005a7fc640 1 -- 172.21.15.33:0/2932722265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00500b0070 con 0x7f005c15b110 2024-04-25T01:09:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005a7fc640 1 -- 172.21.15.33:0/2932722265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00500ab600 con 0x7f005c15b110 2024-04-25T01:09:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f005a7fc640 1 -- 172.21.15.33:0/2932722265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00500ab9d0 con 0x7f005c15b110 2024-04-25T01:09:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 -- 172.21.15.33:0/2932722265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 msgr2=0x7f005c071e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 --2- 172.21.15.33:0/2932722265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c071e50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0050009870 tx=0x7f005001c520 comp rx=0 tx=0).stop 2024-04-25T01:09:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 -- 172.21.15.33:0/2932722265 shutdown_connections 2024-04-25T01:09:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 --2- 172.21.15.33:0/2932722265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c071e50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 --2- 172.21.15.33:0/2932722265 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 -- 172.21.15.33:0/2932722265 >> 172.21.15.33:0/2932722265 conn(0x7f005c0d0af0 msgr2=0x7f005c0d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 -- 172.21.15.33:0/2932722265 shutdown_connections 2024-04-25T01:09:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.660+0000 7f0061f29640 1 -- 172.21.15.33:0/2932722265 wait complete. 2024-04-25T01:09:19.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 Processor -- start 2024-04-25T01:09:19.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 -- start start 2024-04-25T01:09:19.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005c072390 0x7f005c0ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f005c167be0 con 0x7f005c15b110 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f005c167a60 con 0x7f005c072390 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f005c167d60 con 0x7f005c158910 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c0b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60088/0 (socket says 172.21.15.33:60088) 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 -- 172.21.15.33:0/3622926576 learned_addr learned my addr 172.21.15.33:0/3622926576 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005affd640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005b7fe640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005c072390 0x7f005c0ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 -- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 msgr2=0x7f005c1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c1550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 -- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005c072390 msgr2=0x7f005c0ca130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005c072390 0x7f005c0ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 -- 172.21.15.33:0/3622926576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c0b0ed0 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005affd640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f005bfff640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c0b0780 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f005400c7b0 tx=0x7f005400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0043fff640 1 -- 172.21.15.33:0/3622926576 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054004b70 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0043fff640 1 -- 172.21.15.33:0/3622926576 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0054004d10 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0043fff640 1 -- 172.21.15.33:0/3622926576 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005400aa30 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c0b11b0 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c0b1770 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c069040 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.664+0000 7f0043fff640 1 -- 172.21.15.33:0/3622926576 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f005400abd0 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.668+0000 7f0043fff640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00480678c0 0x7f0048069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.668+0000 7f0043fff640 1 -- 172.21.15.33:0/3622926576 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0054086310 con 0x7f005c15b110 2024-04-25T01:09:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.668+0000 7f005b7fe640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00480678c0 0x7f0048069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.668+0000 7f005b7fe640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00480678c0 0x7f0048069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f004c00b800 tx=0x7f004c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.672+0000 7f0043fff640 1 -- 172.21.15.33:0/3622926576 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0054020070 con 0x7f005c15b110 2024-04-25T01:09:19.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.836+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f005c164720 con 0x7f005c15b110 2024-04-25T01:09:19.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.836+0000 7f0043fff640 1 -- 172.21.15.33:0/3622926576 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f0054056a40 con 0x7f005c15b110 2024-04-25T01:09:19.829 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.836+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00480678c0 msgr2=0x7f0048069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.836+0000 7f0061f29640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00480678c0 0x7f0048069d80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f004c00b800 tx=0x7f004c008040 comp rx=0 tx=0).stop 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.836+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 msgr2=0x7f005c0b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.836+0000 7f0061f29640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c0b0780 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f005400c7b0 tx=0x7f005400cc70 comp rx=0 tx=0).stop 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 shutdown_connections 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00480678c0 0x7f0048069d80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f005c15b110 0x7f005c0b0780 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f005c158910 0x7f005c1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 --2- 172.21.15.33:0/3622926576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f005c072390 0x7f005c0ca130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 >> 172.21.15.33:0/3622926576 conn(0x7f005c0d0af0 msgr2=0x7f005c158cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 shutdown_connections 2024-04-25T01:09:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.840+0000 7f0061f29640 1 -- 172.21.15.33:0/3622926576 wait complete. 2024-04-25T01:09:19.849 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:19.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.980+0000 7f47f5624640 1 Processor -- start 2024-04-25T01:09:19.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.980+0000 7f47f5624640 1 -- start start 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.980+0000 7f47f5624640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f47f0145660 0x7f47f006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.980+0000 7f47f5624640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47f015f960 0x7f47f015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.980+0000 7f47f5624640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f00cee10 con 0x7f47f01602e0 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f47f006f0a0 con 0x7f47f015f960 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f47f0145a40 con 0x7f47f0145660 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47e7fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f47f015f960 0x7f47f015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47ef7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f47f01602e0 0x7f47f006d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49056/0 (socket says 172.21.15.33:49056) 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47ef7fe640 1 -- 172.21.15.33:0/1646320615 learned_addr learned my addr 172.21.15.33:0/1646320615 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47eeffd640 1 --2- 172.21.15.33:0/1646320615 >> v2:172.21.15.138:3300/0 conn(0x7f47f0145660 0x7f47f006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47e7fff640 1 -- 172.21.15.33:0/1646320615 >> v2:172.21.15.138:3300/0 conn(0x7f47f0145660 msgr2=0x7f47f006eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:19.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47e7fff640 1 --2- 172.21.15.33:0/1646320615 >> v2:172.21.15.138:3300/0 conn(0x7f47f0145660 0x7f47f006eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47e7fff640 1 -- 172.21.15.33:0/1646320615 >> v1:172.21.15.33:6789/0 conn(0x7f47f01602e0 legacy=0x7f47f006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47e7fff640 1 -- 172.21.15.33:0/1646320615 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f00cf0f0 con 0x7f47f015f960 2024-04-25T01:09:19.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47ee7fc640 1 -- 172.21.15.33:0/1646320615 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1285860008 0 0) 0x7f47f00cee10 con 0x7f47f01602e0 2024-04-25T01:09:19.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47e7fff640 1 --2- 172.21.15.33:0/1646320615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015f960 0x7f47f015fda0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47f006c190 tx=0x7f47d801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3eeefb9c7c03f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47ee7fc640 1 -- 172.21.15.33:0/1646320615 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80b0070 con 0x7f47f015f960 2024-04-25T01:09:19.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47ee7fc640 1 -- 172.21.15.33:0/1646320615 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d80ab600 con 0x7f47f015f960 2024-04-25T01:09:19.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47ee7fc640 1 -- 172.21.15.33:0/1646320615 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d80ab9d0 con 0x7f47f015f960 2024-04-25T01:09:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 -- 172.21.15.33:0/1646320615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015f960 msgr2=0x7f47f015fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 --2- 172.21.15.33:0/1646320615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015f960 0x7f47f015fda0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47f006c190 tx=0x7f47d801c480 comp rx=0 tx=0).stop 2024-04-25T01:09:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 -- 172.21.15.33:0/1646320615 shutdown_connections 2024-04-25T01:09:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 --2- 172.21.15.33:0/1646320615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f015f960 0x7f47f015fda0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 --2- 172.21.15.33:0/1646320615 >> v2:172.21.15.138:3300/0 conn(0x7f47f0145660 0x7f47f006eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 -- 172.21.15.33:0/1646320615 >> 172.21.15.33:0/1646320615 conn(0x7f47f00b1810 msgr2=0x7f47f00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 -- 172.21.15.33:0/1646320615 shutdown_connections 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.984+0000 7f47f5624640 1 -- 172.21.15.33:0/1646320615 wait complete. 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 Processor -- start 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 -- start start 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f0145660 0x7f47f0166200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f015f960 0x7f47f015eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f01602e0 0x7f47f0144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0168e20 con 0x7f47f0145660 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47f0168ca0 con 0x7f47f01602e0 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f47f0168fa0 con 0x7f47f015f960 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f0145660 0x7f47f0166200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f0145660 0x7f47f0166200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60116/0 (socket says 172.21.15.33:60116) 2024-04-25T01:09:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 -- 172.21.15.33:0/1001982133 learned_addr learned my addr 172.21.15.33:0/1001982133 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47e7fff640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f015f960 0x7f47f015eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47ef7fe640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f01602e0 0x7f47f0144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 -- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f015f960 msgr2=0x7f47f015eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f015f960 0x7f47f015eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 -- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f01602e0 msgr2=0x7f47f0144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f01602e0 0x7f47f0144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 -- 172.21.15.33:0/1001982133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f0165160 con 0x7f47f0145660 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47eeffd640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f0145660 0x7f47f0166200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d5f0 tx=0x7f47e000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47e77fe640 1 -- 172.21.15.33:0/1001982133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e00125f0 con 0x7f47f0145660 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47e77fe640 1 -- 172.21.15.33:0/1001982133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47e0019070 con 0x7f47f0145660 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47e77fe640 1 -- 172.21.15.33:0/1001982133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e00129d0 con 0x7f47f0145660 2024-04-25T01:09:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f0165440 con 0x7f47f0145660 2024-04-25T01:09:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.988+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f0165a00 con 0x7f47f0145660 2024-04-25T01:09:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.992+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0069040 con 0x7f47f0145660 2024-04-25T01:09:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.992+0000 7f47e77fe640 1 -- 172.21.15.33:0/1001982133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47e0012b70 con 0x7f47f0145660 2024-04-25T01:09:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.992+0000 7f47e77fe640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47d00678f0 0x7f47d0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.992+0000 7f47e77fe640 1 -- 172.21.15.33:0/1001982133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f47e0086fd0 con 0x7f47f0145660 2024-04-25T01:09:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.992+0000 7f47e7fff640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47d00678f0 0x7f47d0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.992+0000 7f47e7fff640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47d00678f0 0x7f47d0069db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47d80ad000 tx=0x7f47d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:19.996+0000 7f47e77fe640 1 -- 172.21.15.33:0/1001982133 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f47e0057770 con 0x7f47f0145660 2024-04-25T01:09:20.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.152+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f47f0072ac0 con 0x7f47f0145660 2024-04-25T01:09:20.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.152+0000 7f47e77fe640 1 -- 172.21.15.33:0/1001982133 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f47e0057070 con 0x7f47f0145660 2024-04-25T01:09:20.146 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47d00678f0 msgr2=0x7f47d0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47d00678f0 0x7f47d0069db0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f47d80ad000 tx=0x7f47d8008040 comp rx=0 tx=0).stop 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f0145660 msgr2=0x7f47f0166200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f0145660 0x7f47f0166200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d5f0 tx=0x7f47e000dab0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 shutdown_connections 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f47d00678f0 0x7f47d0069db0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47f01602e0 0x7f47f0144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f47f015f960 0x7f47f015eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 --2- 172.21.15.33:0/1001982133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47f0145660 0x7f47f0166200 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 >> 172.21.15.33:0/1001982133 conn(0x7f47f00b1810 msgr2=0x7f47f00cfa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 shutdown_connections 2024-04-25T01:09:20.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.156+0000 7f47f5624640 1 -- 172.21.15.33:0/1001982133 wait complete. 2024-04-25T01:09:20.165 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.292+0000 7fb83f127640 1 Processor -- start 2024-04-25T01:09:20.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.292+0000 7fb83f127640 1 -- start start 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83f127640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb838145660 0x7fb83806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83f127640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb83815f960 0x7fb83815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83f127640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 0x7fb83806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83f127640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb83806f0a0 con 0x7fb83815f960 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83f127640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb838145a40 con 0x7fb838145660 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83f127640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb838070ce0 con 0x7fb8381602e0 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb83815f960 0x7fb83815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb83815f960 0x7fb83815fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60118/0 (socket says 172.21.15.33:60118) 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 -- 172.21.15.33:0/2394075410 learned_addr learned my addr 172.21.15.33:0/2394075410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83cec3640 1 --2- 172.21.15.33:0/2394075410 >> v2:172.21.15.138:3300/0 conn(0x7fb838145660 0x7fb83806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83d6c4640 1 --2- 172.21.15.33:0/2394075410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 0x7fb83806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 -- 172.21.15.33:0/2394075410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 msgr2=0x7fb83806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:20.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 --2- 172.21.15.33:0/2394075410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 0x7fb83806d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 -- 172.21.15.33:0/2394075410 >> v2:172.21.15.138:3300/0 conn(0x7fb838145660 msgr2=0x7fb83806eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 --2- 172.21.15.33:0/2394075410 >> v2:172.21.15.138:3300/0 conn(0x7fb838145660 0x7fb83806eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 -- 172.21.15.33:0/2394075410 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb838160770 con 0x7fb83815f960 2024-04-25T01:09:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb83cec3640 1 --2- 172.21.15.33:0/2394075410 >> v2:172.21.15.138:3300/0 conn(0x7fb838145660 0x7fb83806eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:20.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb837fff640 1 --2- 172.21.15.33:0/2394075410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb83815fda0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c190 tx=0x7fb82801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a171b2ebdad00c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb8377fe640 1 -- 172.21.15.33:0/2394075410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8280b0070 con 0x7fb83815f960 2024-04-25T01:09:20.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb8377fe640 1 -- 172.21.15.33:0/2394075410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8280ab600 con 0x7fb83815f960 2024-04-25T01:09:20.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.296+0000 7fb8377fe640 1 -- 172.21.15.33:0/2394075410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8280ab9d0 con 0x7fb83815f960 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- 172.21.15.33:0/2394075410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 msgr2=0x7fb83815fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 --2- 172.21.15.33:0/2394075410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb83815fda0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb83806c190 tx=0x7fb82801c230 comp rx=0 tx=0).stop 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- 172.21.15.33:0/2394075410 shutdown_connections 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 --2- 172.21.15.33:0/2394075410 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 0x7fb83806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 --2- 172.21.15.33:0/2394075410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb83815fda0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 --2- 172.21.15.33:0/2394075410 >> v2:172.21.15.138:3300/0 conn(0x7fb838145660 0x7fb83806eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- 172.21.15.33:0/2394075410 >> 172.21.15.33:0/2394075410 conn(0x7fb8380b1810 msgr2=0x7fb8380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- 172.21.15.33:0/2394075410 shutdown_connections 2024-04-25T01:09:20.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- 172.21.15.33:0/2394075410 wait complete. 2024-04-25T01:09:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 Processor -- start 2024-04-25T01:09:20.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- start start 2024-04-25T01:09:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838145660 0x7fb8381410d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb8380b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 0x7fb83815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb838071150 con 0x7fb83815f960 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8380cf6b0 con 0x7fb838145660 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb8380712d0 con 0x7fb8381602e0 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb8380b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb8380b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60128/0 (socket says 172.21.15.33:60128) 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 -- 172.21.15.33:0/2144529450 learned_addr learned my addr 172.21.15.33:0/2144529450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 -- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 msgr2=0x7fb83815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 0x7fb83815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 -- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838145660 msgr2=0x7fb8381410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838145660 0x7fb8381410d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 -- 172.21.15.33:0/2144529450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb838141610 con 0x7fb83815f960 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb837fff640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb8380b2f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb8280ae040 tx=0x7fb82801aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb8357fa640 1 -- 172.21.15.33:0/2144529450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8280ba020 con 0x7fb83815f960 2024-04-25T01:09:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb8357fa640 1 -- 172.21.15.33:0/2144529450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8280b6640 con 0x7fb83815f960 2024-04-25T01:09:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8380b38b0 con 0x7fb83815f960 2024-04-25T01:09:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb8357fa640 1 -- 172.21.15.33:0/2144529450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8280b0040 con 0x7fb83815f960 2024-04-25T01:09:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.300+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8380b3e70 con 0x7fb83815f960 2024-04-25T01:09:20.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.304+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8380ad840 con 0x7fb83815f960 2024-04-25T01:09:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.304+0000 7fb8357fa640 1 -- 172.21.15.33:0/2144529450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb8280bd020 con 0x7fb83815f960 2024-04-25T01:09:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.304+0000 7fb8357fa640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8180678c0 0x7fb818069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.304+0000 7fb8357fa640 1 -- 172.21.15.33:0/2144529450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb828120430 con 0x7fb83815f960 2024-04-25T01:09:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.304+0000 7fb83cec3640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8180678c0 0x7fb818069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.304+0000 7fb83cec3640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8180678c0 0x7fb818069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8380b4df0 tx=0x7fb820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.308+0000 7fb8357fa640 1 -- 172.21.15.33:0/2144529450 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb8280c4030 con 0x7fb83815f960 2024-04-25T01:09:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb838069040 con 0x7fb83815f960 2024-04-25T01:09:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb8357fa640 1 -- 172.21.15.33:0/2144529450 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb828016110 con 0x7fb83815f960 2024-04-25T01:09:20.456 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8180678c0 msgr2=0x7fb818069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8180678c0 0x7fb818069d80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8380b4df0 tx=0x7fb820008040 comp rx=0 tx=0).stop 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 msgr2=0x7fb8380b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb8380b2f40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fb8280ae040 tx=0x7fb82801aa70 comp rx=0 tx=0).stop 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 shutdown_connections 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb8180678c0 0x7fb818069d80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8381602e0 0x7fb83815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb83815f960 0x7fb8380b2f40 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 --2- 172.21.15.33:0/2144529450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838145660 0x7fb8381410d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 >> 172.21.15.33:0/2144529450 conn(0x7fb8380b1810 msgr2=0x7fb838142f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:20.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 shutdown_connections 2024-04-25T01:09:20.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.464+0000 7fb83f127640 1 -- 172.21.15.33:0/2144529450 wait complete. 2024-04-25T01:09:20.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:20.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5024fda640 1 Processor -- start 2024-04-25T01:09:20.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5024fda640 1 -- start start 2024-04-25T01:09:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5024fda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200d03e0 0x7f50200d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5024fda640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50200d10b0 con 0x7f502015e470 2024-04-25T01:09:20.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5024fda640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50200d1390 con 0x7f50200d48d0 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5024fda640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50200d1de0 con 0x7f50200d03e0 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5017fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f502015e470 0x7f50200d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49058/0 (socket says 172.21.15.33:49058) 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5017fff640 1 -- 172.21.15.33:0/2485198068 learned_addr learned my addr 172.21.15.33:0/2485198068 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814507903 0 0) 0x7f50200d10b0 con 0x7f502015e470 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f500c003530 con 0x7f502015e470 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501e59c640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200d03e0 0x7f50200d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 280560696 0 0) 0x7f500c003530 con 0x7f502015e470 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200d03e0 msgr2=0x7f50200d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200d03e0 0x7f50200d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 >> v1:172.21.15.138:6789/0 conn(0x7f50200d48d0 legacy=0x7f50200d2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50200d1670 con 0x7f502015e470 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f5008001680 con 0x7f502015e470 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 >> v1:172.21.15.33:6789/0 conn(0x7f502015e470 legacy=0x7f50200d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f500c0039f0 0x7f500c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f500c004810 0x7f500c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f500c005220 0x7f500c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50200d1de0 con 0x7f500c005220 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f500c00c120 con 0x7f500c0039f0 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f500c00c320 con 0x7f500c004810 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501e59c640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200d03e0 0x7f50200d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501ed9d640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f500c005220 0x7f500c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5017fff640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f500c004810 0x7f500c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501e59c640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f500c0039f0 0x7f500c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501ed9d640 1 -- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f500c004810 msgr2=0x7f500c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501ed9d640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f500c004810 0x7f500c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501ed9d640 1 -- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f500c0039f0 msgr2=0x7f500c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501ed9d640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f500c0039f0 0x7f500c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501ed9d640 1 -- 172.21.15.33:0/2485198068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500c003530 con 0x7f500c005220 2024-04-25T01:09:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f5017fff640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f500c004810 0x7f500c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.596+0000 7f501e59c640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f500c0039f0 0x7f500c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f501ed9d640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f500c005220 0x7f500c00b9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5018001e70 tx=0x7f501801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3c2d8460b36625 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018001a90 con 0x7f500c005220 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5018003b80 con 0x7f500c005220 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f501dd9b640 1 -- 172.21.15.33:0/2485198068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018005080 con 0x7f500c005220 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 -- 172.21.15.33:0/2485198068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f500c005220 msgr2=0x7f500c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 --2- 172.21.15.33:0/2485198068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f500c005220 0x7f500c00b9d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5018001e70 tx=0x7f501801c780 comp rx=0 tx=0).stop 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f501ed9d640 1 -- 172.21.15.33:0/2485198068 reap_dead start 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 -- 172.21.15.33:0/2485198068 shutdown_connections 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 -- 172.21.15.33:0/2485198068 >> 172.21.15.33:0/2485198068 conn(0x7f50200d3db0 msgr2=0x7f50200d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 -- 172.21.15.33:0/2485198068 shutdown_connections 2024-04-25T01:09:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 -- 172.21.15.33:0/2485198068 wait complete. 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 Processor -- start 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.600+0000 7f5024fda640 1 -- start start 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f502014cfa0 0x7f50200c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502014d380 0x7f502014d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200c8ac0 0x7f502014de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50200d3210 con 0x7f502014d380 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f50200d3090 con 0x7f502014cfa0 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f50200d3390 con 0x7f50200c8ac0 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502014d380 0x7f502014d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502014d380 0x7f502014d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60140/0 (socket says 172.21.15.33:60140) 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 -- 172.21.15.33:0/2402844601 learned_addr learned my addr 172.21.15.33:0/2402844601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f501e59c640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f502014cfa0 0x7f50200c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f501ed9d640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200c8ac0 0x7f502014de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 -- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200c8ac0 msgr2=0x7f502014de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200c8ac0 0x7f502014de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 -- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f502014cfa0 msgr2=0x7f50200c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f502014cfa0 0x7f50200c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 -- 172.21.15.33:0/2402844601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50200c74e0 con 0x7f502014d380 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f501ed9d640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200c8ac0 0x7f502014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5017fff640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502014d380 0x7f502014d900 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5008010b80 tx=0x7f500800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5016ffd640 1 -- 172.21.15.33:0/2402844601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500801b070 con 0x7f502014d380 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5016ffd640 1 -- 172.21.15.33:0/2402844601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5008011040 con 0x7f502014d380 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50200c77c0 con 0x7f502014d380 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5016ffd640 1 -- 172.21.15.33:0/2402844601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50080143a0 con 0x7f502014d380 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50200c7d80 con 0x7f502014d380 2024-04-25T01:09:20.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5020069040 con 0x7f502014d380 2024-04-25T01:09:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5016ffd640 1 -- 172.21.15.33:0/2402844601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f500801e070 con 0x7f502014d380 2024-04-25T01:09:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5016ffd640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f50000678c0 0x7f5000069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f5016ffd640 1 -- 172.21.15.33:0/2402844601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5008088d70 con 0x7f502014d380 2024-04-25T01:09:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.604+0000 7f501e59c640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f50000678c0 0x7f5000069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.608+0000 7f501e59c640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f50000678c0 0x7f5000069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f502008c7a0 tx=0x7f5010008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.608+0000 7f5016ffd640 1 -- 172.21.15.33:0/2402844601 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5008059410 con 0x7f502014d380 2024-04-25T01:09:20.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.776+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f502006d980 con 0x7f502014d380 2024-04-25T01:09:20.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5016ffd640 1 -- 172.21.15.33:0/2402844601 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5008058d10 con 0x7f502014d380 2024-04-25T01:09:20.772 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f50000678c0 msgr2=0x7f5000069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f50000678c0 0x7f5000069d80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f502008c7a0 tx=0x7f5010008040 comp rx=0 tx=0).stop 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502014d380 msgr2=0x7f502014d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502014d380 0x7f502014d900 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5008010b80 tx=0x7f500800e4d0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 shutdown_connections 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f50000678c0 0x7f5000069d80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f50200c8ac0 0x7f502014de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f502014d380 0x7f502014d900 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 --2- 172.21.15.33:0/2402844601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f502014cfa0 0x7f50200c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 >> 172.21.15.33:0/2402844601 conn(0x7f50200d3db0 msgr2=0x7f5020160f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:20.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 shutdown_connections 2024-04-25T01:09:20.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.780+0000 7f5024fda640 1 -- 172.21.15.33:0/2402844601 wait complete. 2024-04-25T01:09:20.789 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:09:20.789 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:20.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f9072012640 1 Processor -- start 2024-04-25T01:09:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f9072012640 1 -- start start 2024-04-25T01:09:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f9072012640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f906c15c2d0 0x7f906c15c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f9072012640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f906c06b210 0x7f906c06b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f9072012640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f906c0d0a00 con 0x7f906c0d0620 2024-04-25T01:09:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f9072012640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f906c06bbc0 con 0x7f906c06b210 2024-04-25T01:09:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f9072012640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f906c06e3a0 con 0x7f906c15c2d0 2024-04-25T01:09:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f906c06b210 0x7f906c06b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f906c06b210 0x7f906c06b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60146/0 (socket says 172.21.15.33:60146) 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906bfff640 1 -- 172.21.15.33:0/1512635474 learned_addr learned my addr 172.21.15.33:0/1512635474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906affd640 1 --2- 172.21.15.33:0/1512635474 >> v2:172.21.15.138:3300/0 conn(0x7f906c15c2d0 0x7f906c15c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906bfff640 1 -- 172.21.15.33:0/1512635474 >> v2:172.21.15.138:3300/0 conn(0x7f906c15c2d0 msgr2=0x7f906c15c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906bfff640 1 --2- 172.21.15.33:0/1512635474 >> v2:172.21.15.138:3300/0 conn(0x7f906c15c2d0 0x7f906c15c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906bfff640 1 -- 172.21.15.33:0/1512635474 >> v1:172.21.15.138:6789/0 conn(0x7f906c0d0620 legacy=0x7f906c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906bfff640 1 -- 172.21.15.33:0/1512635474 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c15cdc0 con 0x7f906c06b210 2024-04-25T01:09:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.916+0000 7f906affd640 1 --2- 172.21.15.33:0/1512635474 >> v2:172.21.15.138:3300/0 conn(0x7f906c15c2d0 0x7f906c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906bfff640 1 --2- 172.21.15.33:0/1512635474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c06b680 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f906c0aecf0 tx=0x7f906401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897f0ea4433a7ffc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906a7fc640 1 -- 172.21.15.33:0/1512635474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90640a5840 con 0x7f906c06b210 2024-04-25T01:09:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906a7fc640 1 -- 172.21.15.33:0/1512635474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90640a59e0 con 0x7f906c06b210 2024-04-25T01:09:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906a7fc640 1 -- 172.21.15.33:0/1512635474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90640a5db0 con 0x7f906c06b210 2024-04-25T01:09:20.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- 172.21.15.33:0/1512635474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 msgr2=0x7f906c06b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 --2- 172.21.15.33:0/1512635474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c06b680 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f906c0aecf0 tx=0x7f906401c980 comp rx=0 tx=0).stop 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- 172.21.15.33:0/1512635474 shutdown_connections 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 --2- 172.21.15.33:0/1512635474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c06b680 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 --2- 172.21.15.33:0/1512635474 >> v2:172.21.15.138:3300/0 conn(0x7f906c15c2d0 0x7f906c15c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- 172.21.15.33:0/1512635474 >> 172.21.15.33:0/1512635474 conn(0x7f906c0b29c0 msgr2=0x7f906c0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- 172.21.15.33:0/1512635474 shutdown_connections 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- 172.21.15.33:0/1512635474 wait complete. 2024-04-25T01:09:20.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 Processor -- start 2024-04-25T01:09:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- start start 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c0d0620 0x7f906c147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f906c15c2d0 0x7f906c1479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f906c15dca0 con 0x7f906c06b210 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f906c0adf30 con 0x7f906c0d0620 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f906c15de20 con 0x7f906c15c2d0 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c14f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60150/0 (socket says 172.21.15.33:60150) 2024-04-25T01:09:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 -- 172.21.15.33:0/2398593846 learned_addr learned my addr 172.21.15.33:0/2398593846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 -- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f906c15c2d0 msgr2=0x7f906c1479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f906c15c2d0 0x7f906c1479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 -- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c0d0620 msgr2=0x7f906c147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c0d0620 0x7f906c147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 -- 172.21.15.33:0/2398593846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906c09bd40 con 0x7f906c06b210 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f906b7fe640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c14f9e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c450 tx=0x7f905c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9053fff640 1 -- 172.21.15.33:0/2398593846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c012830 con 0x7f906c06b210 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9053fff640 1 -- 172.21.15.33:0/2398593846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f905c017070 con 0x7f906c06b210 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906c09c020 con 0x7f906c06b210 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9053fff640 1 -- 172.21.15.33:0/2398593846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f905c012c30 con 0x7f906c06b210 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.920+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f906c09cb00 con 0x7f906c06b210 2024-04-25T01:09:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.924+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f906c069040 con 0x7f906c06b210 2024-04-25T01:09:20.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.924+0000 7f9053fff640 1 -- 172.21.15.33:0/2398593846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f905c01c460 con 0x7f906c06b210 2024-04-25T01:09:20.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.924+0000 7f9053fff640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90580678f0 0x7f9058069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:20.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.924+0000 7f9053fff640 1 -- 172.21.15.33:0/2398593846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f905c0868d0 con 0x7f906c06b210 2024-04-25T01:09:20.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.924+0000 7f906affd640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90580678f0 0x7f9058069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:20.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.924+0000 7f906affd640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90580678f0 0x7f9058069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f906c1f7cd0 tx=0x7f9060008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:20.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:20.928+0000 7f9053fff640 1 -- 172.21.15.33:0/2398593846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f905c0570f0 con 0x7f906c06b210 2024-04-25T01:09:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.088+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f906c09b150 con 0x7f906c06b210 2024-04-25T01:09:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9053fff640 1 -- 172.21.15.33:0/2398593846 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7f905c0569f0 con 0x7f906c06b210 2024-04-25T01:09:21.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:21.084 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90580678f0 msgr2=0x7f9058069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90580678f0 0x7f9058069db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f906c1f7cd0 tx=0x7f9060008040 comp rx=0 tx=0).stop 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 msgr2=0x7f906c14f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c14f9e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f905c00c450 tx=0x7f905c00c910 comp rx=0 tx=0).stop 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 shutdown_connections 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90580678f0 0x7f9058069db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f906c15c2d0 0x7f906c1479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c0d0620 0x7f906c147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 --2- 172.21.15.33:0/2398593846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c06b210 0x7f906c14f9e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 >> 172.21.15.33:0/2398593846 conn(0x7f906c0b29c0 msgr2=0x7f906c0b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 shutdown_connections 2024-04-25T01:09:21.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.092+0000 7f9072012640 1 -- 172.21.15.33:0/2398593846 wait complete. 2024-04-25T01:09:21.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:21.104 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb01b2b5640 1 Processor -- start 2024-04-25T01:09:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb01b2b5640 1 -- start start 2024-04-25T01:09:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb01b2b5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 0x7fb0140b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb01b2b5640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0140b4a20 con 0x7fb01415ae20 2024-04-25T01:09:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb01b2b5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0140b2bb0 con 0x7fb01406ecc0 2024-04-25T01:09:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb01b2b5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0140b2e90 con 0x7fb014072410 2024-04-25T01:09:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019051640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb01415ae20 0x7fb01406e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49068/0 (socket says 172.21.15.33:49068) 2024-04-25T01:09:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019051640 1 -- 172.21.15.33:0/1910885505 learned_addr learned my addr 172.21.15.33:0/1910885505 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2801463000 0 0) 0x7fb0140b4a20 con 0x7fb01415ae20 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019852640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 0x7fb0140b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb000005740 con 0x7fb01415ae20 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 165336705 0 0) 0x7fb000005740 con 0x7fb01415ae20 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 msgr2=0x7fb0140b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 0x7fb0140b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 >> v1:172.21.15.138:6789/0 conn(0x7fb01406ecc0 legacy=0x7fb0140cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0140b3170 con 0x7fb01415ae20 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fb010001680 con 0x7fb01415ae20 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 >> v1:172.21.15.33:6789/0 conn(0x7fb01415ae20 legacy=0x7fb01406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb000005c00 0x7fb000006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 0x7fb000006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb000006f60 0x7fb0000073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140b2e90 con 0x7fb000006f60 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb00000dd70 con 0x7fb000005c00 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb00000df40 con 0x7fb014072410 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019852640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb000006f60 0x7fb0000073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb018850640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 0x7fb000006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019051640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb000005c00 0x7fb000006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019852640 1 -- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 msgr2=0x7fb000006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019852640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb014072410 0x7fb000006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019852640 1 -- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb000005c00 msgr2=0x7fb000006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019852640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb000005c00 0x7fb000006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019852640 1 -- 172.21.15.33:0/1910885505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb000005740 con 0x7fb000006f60 2024-04-25T01:09:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.228+0000 7fb019051640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb000005c00 0x7fb000006020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb019852640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb000006f60 0x7fb0000073e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007360 tx=0x7fb00c01ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8234d188acd5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c0b2070 con 0x7fb000006f60 2024-04-25T01:09:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb00c0a5d00 con 0x7fb000006f60 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb00bfff640 1 -- 172.21.15.33:0/1910885505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00c009be0 con 0x7fb000006f60 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 -- 172.21.15.33:0/1910885505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb000006f60 msgr2=0x7fb0000073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 --2- 172.21.15.33:0/1910885505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb000006f60 0x7fb0000073e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007360 tx=0x7fb00c01ce00 comp rx=0 tx=0).stop 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb019852640 1 -- 172.21.15.33:0/1910885505 reap_dead start 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 -- 172.21.15.33:0/1910885505 shutdown_connections 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 -- 172.21.15.33:0/1910885505 >> 172.21.15.33:0/1910885505 conn(0x7fb0140b1810 msgr2=0x7fb0140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 -- 172.21.15.33:0/1910885505 shutdown_connections 2024-04-25T01:09:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 -- 172.21.15.33:0/1910885505 wait complete. 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 Processor -- start 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.232+0000 7fb01b2b5640 1 -- start start 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb014150c80 0x7fb0140a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb014151060 0x7fb01415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb01415b860 0x7fb01415bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140b3690 con 0x7fb014151060 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb0140716f0 con 0x7fb014150c80 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb0140b3810 con 0x7fb01415b860 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb014151060 0x7fb01415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb014151060 0x7fb01415d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60168/0 (socket says 172.21.15.33:60168) 2024-04-25T01:09:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 -- 172.21.15.33:0/2372688687 learned_addr learned my addr 172.21.15.33:0/2372688687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb019051640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb014150c80 0x7fb0140a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 -- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb01415b860 msgr2=0x7fb01415bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb019852640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb01415b860 0x7fb01415bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb01415b860 0x7fb01415bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 -- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb014150c80 msgr2=0x7fb0140a9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb014150c80 0x7fb0140a9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 -- 172.21.15.33:0/2372688687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb014152300 con 0x7fb014151060 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb019852640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb01415b860 0x7fb01415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb018850640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb014151060 0x7fb01415d9f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb00400f820 tx=0x7fb00400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb009ffb640 1 -- 172.21.15.33:0/2372688687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0040050d0 con 0x7fb014151060 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb009ffb640 1 -- 172.21.15.33:0/2372688687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb004018070 con 0x7fb014151060 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0141525e0 con 0x7fb014151060 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb009ffb640 1 -- 172.21.15.33:0/2372688687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00400d730 con 0x7fb014151060 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb014152ba0 con 0x7fb014151060 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb014069040 con 0x7fb014151060 2024-04-25T01:09:21.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb009ffb640 1 -- 172.21.15.33:0/2372688687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb00400d8d0 con 0x7fb014151060 2024-04-25T01:09:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.236+0000 7fb009ffb640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff80678c0 0x7faff8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.240+0000 7fb009ffb640 1 -- 172.21.15.33:0/2372688687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb004085f80 con 0x7fb014151060 2024-04-25T01:09:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.240+0000 7fb019051640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff80678c0 0x7faff8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.240+0000 7fb019051640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff80678c0 0x7faff8069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb014155b80 tx=0x7fb010009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.244+0000 7fb009ffb640 1 -- 172.21.15.33:0/2372688687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb004056620 con 0x7fb014151060 2024-04-25T01:09:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb01406ce50 con 0x7fb014151060 2024-04-25T01:09:21.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb009ffb640 1 -- 172.21.15.33:0/2372688687 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fb004055f20 con 0x7fb014151060 2024-04-25T01:09:21.392 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff80678c0 msgr2=0x7faff8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff80678c0 0x7faff8069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb014155b80 tx=0x7fb010009040 comp rx=0 tx=0).stop 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb014151060 msgr2=0x7fb01415d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb014151060 0x7fb01415d9f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb00400f820 tx=0x7fb00400fce0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 shutdown_connections 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faff80678c0 0x7faff8069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb01415b860 0x7fb01415bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb014151060 0x7fb01415d9f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 --2- 172.21.15.33:0/2372688687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb014150c80 0x7fb0140a9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 >> 172.21.15.33:0/2372688687 conn(0x7fb0140b1810 msgr2=0x7fb0140af6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 shutdown_connections 2024-04-25T01:09:21.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.400+0000 7fb01b2b5640 1 -- 172.21.15.33:0/2372688687 wait complete. 2024-04-25T01:09:21.410 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3fe44640 1 Processor -- start 2024-04-25T01:09:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3fe44640 1 -- start start 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3fe44640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c3806b210 0x7f0c380b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3fe44640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0c380b0400 0x7f0c380b0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3fe44640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c380b55a0 con 0x7f0c38067030 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3fe44640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0c380b61b0 con 0x7f0c3806b210 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3fe44640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f0c380b5880 con 0x7f0c380b0400 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3dbe0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c3806b210 0x7f0c380b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3d3df640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0c38067030 0x7f0c3806b5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49070/0 (socket says 172.21.15.33:49070) 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3d3df640 1 -- 172.21.15.33:0/682014055 learned_addr learned my addr 172.21.15.33:0/682014055 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3dbe0640 1 -- 172.21.15.33:0/682014055 >> v2:172.21.15.138:3300/0 conn(0x7f0c380b0400 msgr2=0x7f0c380b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3dbe0640 1 --2- 172.21.15.33:0/682014055 >> v2:172.21.15.138:3300/0 conn(0x7f0c380b0400 0x7f0c380b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3dbe0640 1 -- 172.21.15.33:0/682014055 >> v1:172.21.15.33:6789/0 conn(0x7f0c38067030 legacy=0x7f0c3806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.544+0000 7f0c3dbe0640 1 -- 172.21.15.33:0/682014055 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c380b5b60 con 0x7f0c3806b210 2024-04-25T01:09:21.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3dbe0640 1 --2- 172.21.15.33:0/682014055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c380b4fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0c380b0090 tx=0x7f0c3001c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748c6f8a0dad9baf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3cbde640 1 -- 172.21.15.33:0/682014055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300b2070 con 0x7f0c3806b210 2024-04-25T01:09:21.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3cbde640 1 -- 172.21.15.33:0/682014055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c300a58b0 con 0x7f0c3806b210 2024-04-25T01:09:21.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3cbde640 1 -- 172.21.15.33:0/682014055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300a5c80 con 0x7f0c3806b210 2024-04-25T01:09:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- 172.21.15.33:0/682014055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 msgr2=0x7f0c380b4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/682014055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c380b4fc0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0c380b0090 tx=0x7f0c3001c840 comp rx=0 tx=0).stop 2024-04-25T01:09:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- 172.21.15.33:0/682014055 shutdown_connections 2024-04-25T01:09:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/682014055 >> v2:172.21.15.138:3300/0 conn(0x7f0c380b0400 0x7f0c380b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/682014055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c380b4fc0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- 172.21.15.33:0/682014055 >> 172.21.15.33:0/682014055 conn(0x7f0c380b1e60 msgr2=0x7f0c380b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:21.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- 172.21.15.33:0/682014055 shutdown_connections 2024-04-25T01:09:21.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- 172.21.15.33:0/682014055 wait complete. 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 Processor -- start 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- start start 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c38067030 0x7f0c3814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c3815df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380b0400 0x7f0c3815e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38067590 con 0x7f0c3806b210 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c38067410 con 0x7f0c380b0400 2024-04-25T01:09:21.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3fe44640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3816a930 con 0x7f0c38067030 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3d3df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c3815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3d3df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c3815df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60192/0 (socket says 172.21.15.33:60192) 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3d3df640 1 -- 172.21.15.33:0/3227668249 learned_addr learned my addr 172.21.15.33:0/3227668249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3dbe0640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c38067030 0x7f0c3814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.548+0000 7f0c3e3e1640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380b0400 0x7f0c3815e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3d3df640 1 -- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c38067030 msgr2=0x7f0c3814ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3d3df640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c38067030 0x7f0c3814ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3d3df640 1 -- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380b0400 msgr2=0x7f0c3815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3d3df640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380b0400 0x7f0c3815e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3d3df640 1 -- 172.21.15.33:0/3227668249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c38161370 con 0x7f0c3806b210 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3d3df640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c3815df70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c420 tx=0x7f0c2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c2e7fc640 1 -- 172.21.15.33:0/3227668249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c28017070 con 0x7f0c3806b210 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c2e7fc640 1 -- 172.21.15.33:0/3227668249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c28005cf0 con 0x7f0c3806b210 2024-04-25T01:09:21.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c2e7fc640 1 -- 172.21.15.33:0/3227668249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c280072f0 con 0x7f0c3806b210 2024-04-25T01:09:21.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c38161650 con 0x7f0c3806b210 2024-04-25T01:09:21.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c3815fa00 con 0x7f0c3806b210 2024-04-25T01:09:21.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c38069040 con 0x7f0c3806b210 2024-04-25T01:09:21.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c2e7fc640 1 -- 172.21.15.33:0/3227668249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0c2801a070 con 0x7f0c3806b210 2024-04-25T01:09:21.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c2e7fc640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c1c0678c0 0x7f0c1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c2e7fc640 1 -- 172.21.15.33:0/3227668249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0c280853a0 con 0x7f0c3806b210 2024-04-25T01:09:21.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3dbe0640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c1c0678c0 0x7f0c1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.552+0000 7f0c3dbe0640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c1c0678c0 0x7f0c1c069d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c300af000 tx=0x7f0c300a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.556+0000 7f0c2e7fc640 1 -- 172.21.15.33:0/3227668249 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0c280554a0 con 0x7f0c3806b210 2024-04-25T01:09:21.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.716+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c380cf1e0 con 0x7f0c3806b210 2024-04-25T01:09:21.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.716+0000 7f0c2e7fc640 1 -- 172.21.15.33:0/3227668249 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f0c280597a0 con 0x7f0c3806b210 2024-04-25T01:09:21.710 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:21.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c1c0678c0 msgr2=0x7f0c1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c1c0678c0 0x7f0c1c069d80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c300af000 tx=0x7f0c300a7070 comp rx=0 tx=0).stop 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 msgr2=0x7f0c3815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c3815df70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0c2800c420 tx=0x7f0c2800c8e0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 shutdown_connections 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c1c0678c0 0x7f0c1c069d80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380b0400 0x7f0c3815e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c3806b210 0x7f0c3815df70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 --2- 172.21.15.33:0/3227668249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c38067030 0x7f0c3814ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 >> 172.21.15.33:0/3227668249 conn(0x7f0c380b1e60 msgr2=0x7f0c380cb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 shutdown_connections 2024-04-25T01:09:21.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.720+0000 7f0c3fe44640 1 -- 172.21.15.33:0/3227668249 wait complete. 2024-04-25T01:09:21.730 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:21.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.864+0000 7fa2a5fd6640 1 Processor -- start 2024-04-25T01:09:21.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.864+0000 7fa2a5fd6640 1 -- start start 2024-04-25T01:09:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa2a5fd6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a015e0a0 0x7fa2a00716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa2a5fd6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2a00b31a0 0x7fa2a00ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa2a5fd6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a015e480 con 0x7fa2a00aec20 2024-04-25T01:09:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa2a5fd6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a00ae220 con 0x7fa2a00b31a0 2024-04-25T01:09:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa2a5fd6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a006e880 con 0x7fa2a015e0a0 2024-04-25T01:09:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29effd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa2a00b31a0 0x7fa2a00ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29ffff640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa2a00aec20 0x7fa2a0161bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:53268/0 (socket says 172.21.15.33:53268) 2024-04-25T01:09:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29f7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a015e0a0 0x7fa2a00716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29ffff640 1 -- 172.21.15.33:0/1418859474 learned_addr learned my addr 172.21.15.33:0/1418859474 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29e7fc640 1 -- 172.21.15.33:0/1418859474 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3066441099 0 0) 0x7fa2a015e480 con 0x7fa2a00aec20 2024-04-25T01:09:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29e7fc640 1 -- 172.21.15.33:0/1418859474 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa28c003530 con 0x7fa2a00aec20 2024-04-25T01:09:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29f7fe640 1 -- 172.21.15.33:0/1418859474 >> v2:172.21.15.138:3300/0 conn(0x7fa2a00b31a0 msgr2=0x7fa2a00ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29f7fe640 1 --2- 172.21.15.33:0/1418859474 >> v2:172.21.15.138:3300/0 conn(0x7fa2a00b31a0 0x7fa2a00ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29f7fe640 1 -- 172.21.15.33:0/1418859474 >> v1:172.21.15.138:6789/0 conn(0x7fa2a00aec20 legacy=0x7fa2a0161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.868+0000 7fa29f7fe640 1 -- 172.21.15.33:0/1418859474 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a006df00 con 0x7fa2a015e0a0 2024-04-25T01:09:21.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa29f7fe640 1 --2- 172.21.15.33:0/1418859474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a015e0a0 0x7fa2a00716a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa2880097c0 tx=0x7fa28801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faad722937920fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa29e7fc640 1 -- 172.21.15.33:0/1418859474 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2880b0070 con 0x7fa2a015e0a0 2024-04-25T01:09:21.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa29e7fc640 1 -- 172.21.15.33:0/1418859474 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2880ab600 con 0x7fa2a015e0a0 2024-04-25T01:09:21.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa29e7fc640 1 -- 172.21.15.33:0/1418859474 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2880ab9b0 con 0x7fa2a015e0a0 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/1418859474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a015e0a0 msgr2=0x7fa2a00716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/1418859474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a015e0a0 0x7fa2a00716a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa2880097c0 tx=0x7fa28801c410 comp rx=0 tx=0).stop 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/1418859474 shutdown_connections 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/1418859474 >> v2:172.21.15.138:3300/0 conn(0x7fa2a00b31a0 0x7fa2a00ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/1418859474 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a015e0a0 0x7fa2a00716a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/1418859474 >> 172.21.15.33:0/1418859474 conn(0x7fa2a00b1810 msgr2=0x7fa2a00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/1418859474 shutdown_connections 2024-04-25T01:09:21.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/1418859474 wait complete. 2024-04-25T01:09:21.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 Processor -- start 2024-04-25T01:09:21.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.872+0000 7fa2a5fd6640 1 -- start start 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a00aec20 0x7fa2a0160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a00b31a0 0x7fa2a00b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a015e0a0 0x7fa2a01554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a016af70 con 0x7fa2a015e0a0 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00af240 con 0x7fa2a00aec20 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a016b0f0 con 0x7fa2a00b31a0 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a015e0a0 0x7fa2a01554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a015e0a0 0x7fa2a01554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60204/0 (socket says 172.21.15.33:60204) 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 -- 172.21.15.33:0/3055377164 learned_addr learned my addr 172.21.15.33:0/3055377164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29effd640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a00b31a0 0x7fa2a00b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29f7fe640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a00aec20 0x7fa2a0160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 -- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a00b31a0 msgr2=0x7fa2a00b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a00b31a0 0x7fa2a00b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 -- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a00aec20 msgr2=0x7fa2a0160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a00aec20 0x7fa2a0160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 -- 172.21.15.33:0/3055377164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01496c0 con 0x7fa2a015e0a0 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29ffff640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a015e0a0 0x7fa2a01554a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa29800caa0 tx=0x7fa29800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa297fff640 1 -- 172.21.15.33:0/3055377164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298017070 con 0x7fa2a015e0a0 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa297fff640 1 -- 172.21.15.33:0/3055377164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29800ac00 con 0x7fa2a015e0a0 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa297fff640 1 -- 172.21.15.33:0/3055377164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298012400 con 0x7fa2a015e0a0 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a00ce7f0 con 0x7fa2a015e0a0 2024-04-25T01:09:21.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a00cedb0 con 0x7fa2a015e0a0 2024-04-25T01:09:21.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a0069040 con 0x7fa2a015e0a0 2024-04-25T01:09:21.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa297fff640 1 -- 172.21.15.33:0/3055377164 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa29801a070 con 0x7fa2a015e0a0 2024-04-25T01:09:21.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa297fff640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa2800678c0 0x7fa280069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:21.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa297fff640 1 -- 172.21.15.33:0/3055377164 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa298085d80 con 0x7fa2a015e0a0 2024-04-25T01:09:21.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.876+0000 7fa29f7fe640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa2800678c0 0x7fa280069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:21.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.880+0000 7fa29f7fe640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa2800678c0 0x7fa280069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2880ad000 tx=0x7fa2880a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:21.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:21.880+0000 7fa297fff640 1 -- 172.21.15.33:0/3055377164 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa298056550 con 0x7fa2a015e0a0 2024-04-25T01:09:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa2a00b2d50 con 0x7fa2a015e0a0 2024-04-25T01:09:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa297fff640 1 -- 172.21.15.33:0/3055377164 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa298055e50 con 0x7fa2a015e0a0 2024-04-25T01:09:22.040 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa2800678c0 msgr2=0x7fa280069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa2800678c0 0x7fa280069d80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2880ad000 tx=0x7fa2880a6000 comp rx=0 tx=0).stop 2024-04-25T01:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a015e0a0 msgr2=0x7fa2a01554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a015e0a0 0x7fa2a01554a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa29800caa0 tx=0x7fa29800cf60 comp rx=0 tx=0).stop 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 shutdown_connections 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa2800678c0 0x7fa280069d80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2a015e0a0 0x7fa2a01554a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa2a00b31a0 0x7fa2a00b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 --2- 172.21.15.33:0/3055377164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a00aec20 0x7fa2a0160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 >> 172.21.15.33:0/3055377164 conn(0x7fa2a00b1810 msgr2=0x7fa2a0156830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 shutdown_connections 2024-04-25T01:09:22.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.048+0000 7fa2a5fd6640 1 -- 172.21.15.33:0/3055377164 wait complete. 2024-04-25T01:09:22.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:22.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fd000602640 1 Processor -- start 2024-04-25T01:09:22.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fd000602640 1 -- start start 2024-04-25T01:09:22.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fd000602640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcff8166f30 0x7fcff8169320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fd000602640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff80d29f0 con 0x7fcff8161720 2024-04-25T01:09:22.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fd000602640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcff80ceae0 con 0x7fcff80d23c0 2024-04-25T01:09:22.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fd000602640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fcff80cedc0 con 0x7fcff8166f30 2024-04-25T01:09:22.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fcffe39e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fcff8161720 0x7fcff80cf5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49076/0 (socket says 172.21.15.33:49076) 2024-04-25T01:09:22.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fcffe39e640 1 -- 172.21.15.33:0/462911132 learned_addr learned my addr 172.21.15.33:0/462911132 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:22.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fcffeb9f640 1 --2- 172.21.15.33:0/462911132 >> v2:172.21.15.33:3300/0 conn(0x7fcff8166f30 0x7fcff8169320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fcffeb9f640 1 -- 172.21.15.33:0/462911132 >> v1:172.21.15.138:6789/0 conn(0x7fcff80d23c0 legacy=0x7fcff80b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:22.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fcffd39c640 1 -- 172.21.15.33:0/462911132 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333977446 0 0) 0x7fcff80d29f0 con 0x7fcff8161720 2024-04-25T01:09:22.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fcffeb9f640 1 -- 172.21.15.33:0/462911132 >> v1:172.21.15.33:6789/0 conn(0x7fcff8161720 legacy=0x7fcff80cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.192+0000 7fcffeb9f640 1 -- 172.21.15.33:0/462911132 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff80cf0a0 con 0x7fcff8166f30 2024-04-25T01:09:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffeb9f640 1 --2- 172.21.15.33:0/462911132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff8169320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcff00097c0 tx=0x7fcff001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18aadb5e8fd6dfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffd39c640 1 -- 172.21.15.33:0/462911132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00b0070 con 0x7fcff8166f30 2024-04-25T01:09:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffd39c640 1 -- 172.21.15.33:0/462911132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff00ab600 con 0x7fcff8166f30 2024-04-25T01:09:22.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffd39c640 1 -- 172.21.15.33:0/462911132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00ab9d0 con 0x7fcff8166f30 2024-04-25T01:09:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- 172.21.15.33:0/462911132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 msgr2=0x7fcff8169320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 --2- 172.21.15.33:0/462911132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff8169320 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcff00097c0 tx=0x7fcff001c4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- 172.21.15.33:0/462911132 shutdown_connections 2024-04-25T01:09:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 --2- 172.21.15.33:0/462911132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff8169320 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- 172.21.15.33:0/462911132 >> 172.21.15.33:0/462911132 conn(0x7fcff80d3e30 msgr2=0x7fcff80d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- 172.21.15.33:0/462911132 shutdown_connections 2024-04-25T01:09:22.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- 172.21.15.33:0/462911132 wait complete. 2024-04-25T01:09:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 Processor -- start 2024-04-25T01:09:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- start start 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff80d23c0 0x7fcff80cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcff8161720 0x7fcff815b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff815f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcff80b3ef0 con 0x7fcff8166f30 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcff80b3d70 con 0x7fcff80d23c0 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fd000602640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcff816c490 con 0x7fcff8161720 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffeb9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff815f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffeb9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff815f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60218/0 (socket says 172.21.15.33:60218) 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffdb9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcff8161720 0x7fcff815b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffe39e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff80d23c0 0x7fcff80cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.196+0000 7fcffeb9f640 1 -- 172.21.15.33:0/3071358539 learned_addr learned my addr 172.21.15.33:0/3071358539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffeb9f640 1 -- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcff8161720 msgr2=0x7fcff815b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffeb9f640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcff8161720 0x7fcff815b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffeb9f640 1 -- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff80d23c0 msgr2=0x7fcff80cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffeb9f640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff80d23c0 0x7fcff80cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffeb9f640 1 -- 172.21.15.33:0/3071358539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff80b73c0 con 0x7fcff8166f30 2024-04-25T01:09:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffeb9f640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff815f490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcff00190b0 tx=0x7fcff001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcfeeffd640 1 -- 172.21.15.33:0/3071358539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00b0050 con 0x7fcff8166f30 2024-04-25T01:09:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcfeeffd640 1 -- 172.21.15.33:0/3071358539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff00bc070 con 0x7fcff8166f30 2024-04-25T01:09:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff80b7680 con 0x7fcff8166f30 2024-04-25T01:09:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcfeeffd640 1 -- 172.21.15.33:0/3071358539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff00b5850 con 0x7fcff8166f30 2024-04-25T01:09:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff80b7c40 con 0x7fcff8166f30 2024-04-25T01:09:22.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff8069040 con 0x7fcff8166f30 2024-04-25T01:09:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcfeeffd640 1 -- 172.21.15.33:0/3071358539 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcff00b59f0 con 0x7fcff8166f30 2024-04-25T01:09:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcfeeffd640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfdc0678c0 0x7fcfdc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcfeeffd640 1 -- 172.21.15.33:0/3071358539 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcff0120880 con 0x7fcff8166f30 2024-04-25T01:09:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffe39e640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfdc0678c0 0x7fcfdc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.200+0000 7fcffe39e640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfdc0678c0 0x7fcfdc069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcff4001570 tx=0x7fcff40014c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.204+0000 7fcfeeffd640 1 -- 172.21.15.33:0/3071358539 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcff0016090 con 0x7fcff8166f30 2024-04-25T01:09:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.364+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcff808db80 con 0x7fcff8166f30 2024-04-25T01:09:22.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fcfeeffd640 1 -- 172.21.15.33:0/3071358539 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fcff00f0970 con 0x7fcff8166f30 2024-04-25T01:09:22.359 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfdc0678c0 msgr2=0x7fcfdc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfdc0678c0 0x7fcfdc069d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fcff4001570 tx=0x7fcff40014c0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 msgr2=0x7fcff815f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff815f490 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fcff00190b0 tx=0x7fcff001ace0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 shutdown_connections 2024-04-25T01:09:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfdc0678c0 0x7fcfdc069d80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcff8166f30 0x7fcff815f490 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcff8161720 0x7fcff815b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 --2- 172.21.15.33:0/3071358539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcff80d23c0 0x7fcff80cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 >> 172.21.15.33:0/3071358539 conn(0x7fcff80d3e30 msgr2=0x7fcff81623f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 shutdown_connections 2024-04-25T01:09:22.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.368+0000 7fd000602640 1 -- 172.21.15.33:0/3071358539 wait complete. 2024-04-25T01:09:22.380 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:22.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.512+0000 7f8d8df3c640 1 Processor -- start 2024-04-25T01:09:22.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.512+0000 7f8d8df3c640 1 -- start start 2024-04-25T01:09:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d8df3c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 0x7f8d8806bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d8df3c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d8815e950 con 0x7f8d8815e570 2024-04-25T01:09:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d8df3c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d880b5e90 con 0x7f8d880cd710 2024-04-25T01:09:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d8df3c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d880b6170 con 0x7f8d880d08f0 2024-04-25T01:09:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d877fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8d8815e570 0x7f8d880ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49086/0 (socket says 172.21.15.33:49086) 2024-04-25T01:09:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d877fe640 1 -- 172.21.15.33:0/276246748 learned_addr learned my addr 172.21.15.33:0/276246748 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:22.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 0x7f8d8806bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977535301 0 0) 0x7f8d8815e950 con 0x7f8d8815e570 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d74003530 con 0x7f8d8815e570 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 986504906 0 0) 0x7f8d74003530 con 0x7f8d8815e570 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 msgr2=0x7f8d8806bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 0x7f8d8806bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 >> v1:172.21.15.138:6789/0 conn(0x7f8d880cd710 legacy=0x7f8d880d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d880670f0 con 0x7f8d8815e570 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f8d78001680 con 0x7f8d8815e570 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 >> v1:172.21.15.33:6789/0 conn(0x7f8d8815e570 legacy=0x7f8d880ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 0x7f8d740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d74003f70 0x7f8d74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d74004d50 0x7f8d740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d880b6170 con 0x7f8d74004d50 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d7400bb60 con 0x7f8d74003f70 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d7400df70 con 0x7f8d880d08f0 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d74004d50 0x7f8d740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d877fe640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 0x7f8d740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d7ffff640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d74003f70 0x7f8d74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 -- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 msgr2=0x7f8d740039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d880d08f0 0x7f8d740039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 -- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d74003f70 msgr2=0x7f8d74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d74003f70 0x7f8d74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 -- 172.21.15.33:0/276246748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d74003530 con 0x7f8d74004d50 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.516+0000 7f8d87fff640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d74004d50 0x7f8d740051d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009f90 tx=0x7f8d8001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b2b96c528cd524 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800b2070 con 0x7f8d74004d50 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d800a5b40 con 0x7f8d74004d50 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d86ffd640 1 -- 172.21.15.33:0/276246748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80009c10 con 0x7f8d74004d50 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- 172.21.15.33:0/276246748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d74004d50 msgr2=0x7f8d740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 --2- 172.21.15.33:0/276246748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d74004d50 0x7f8d740051d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009f90 tx=0x7f8d8001cc40 comp rx=0 tx=0).stop 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 -- 172.21.15.33:0/276246748 reap_dead start 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- 172.21.15.33:0/276246748 shutdown_connections 2024-04-25T01:09:22.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- 172.21.15.33:0/276246748 >> 172.21.15.33:0/276246748 conn(0x7f8d880b50c0 msgr2=0x7f8d880b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- 172.21.15.33:0/276246748 shutdown_connections 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- 172.21.15.33:0/276246748 wait complete. 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 Processor -- start 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- start start 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8814b370 0x7f8d8815ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8814b750 0x7f8d8814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d88153f90 0x7f8d88154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8806bbe0 con 0x7f8d88153f90 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8806ba60 con 0x7f8d8814b750 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8d8806b330 con 0x7f8d8814b370 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d88153f90 0x7f8d88154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d88153f90 0x7f8d88154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60226/0 (socket says 172.21.15.33:60226) 2024-04-25T01:09:22.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 -- 172.21.15.33:0/3112553446 learned_addr learned my addr 172.21.15.33:0/3112553446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d877fe640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8814b370 0x7f8d8815ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d7ffff640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8814b750 0x7f8d8814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 -- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8814b370 msgr2=0x7f8d8815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8814b370 0x7f8d8815ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 -- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8814b750 msgr2=0x7f8d8814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8814b750 0x7f8d8814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 -- 172.21.15.33:0/3112553446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8814d6e0 con 0x7f8d88153f90 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d7ffff640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8814b750 0x7f8d8814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d87fff640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d88153f90 0x7f8d88154410 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8d80002240 tx=0x7f8d80009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d84ff9640 1 -- 172.21.15.33:0/3112553446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800bd070 con 0x7f8d88153f90 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d84ff9640 1 -- 172.21.15.33:0/3112553446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d800a5940 con 0x7f8d88153f90 2024-04-25T01:09:22.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8814d9c0 con 0x7f8d88153f90 2024-04-25T01:09:22.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d84ff9640 1 -- 172.21.15.33:0/3112553446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d800b2040 con 0x7f8d88153f90 2024-04-25T01:09:22.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.520+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8814df80 con 0x7f8d88153f90 2024-04-25T01:09:22.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.524+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d88154950 con 0x7f8d88153f90 2024-04-25T01:09:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.524+0000 7f8d84ff9640 1 -- 172.21.15.33:0/3112553446 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8d800c0070 con 0x7f8d88153f90 2024-04-25T01:09:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.524+0000 7f8d84ff9640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d680678f0 0x7f8d68069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.524+0000 7f8d84ff9640 1 -- 172.21.15.33:0/3112553446 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8d8011fe20 con 0x7f8d88153f90 2024-04-25T01:09:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.524+0000 7f8d877fe640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d680678f0 0x7f8d68069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:22.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.524+0000 7f8d877fe640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d680678f0 0x7f8d68069db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c10 tx=0x7f8d78001530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:22.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.528+0000 7f8d84ff9640 1 -- 172.21.15.33:0/3112553446 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8d80016950 con 0x7f8d88153f90 2024-04-25T01:09:22.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.704+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8d880ceb20 con 0x7f8d88153f90 2024-04-25T01:09:22.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.704+0000 7f8d84ff9640 1 -- 172.21.15.33:0/3112553446 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f8d800f0060 con 0x7f8d88153f90 2024-04-25T01:09:22.698 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d680678f0 msgr2=0x7f8d68069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d680678f0 0x7f8d68069db0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c10 tx=0x7f8d78001530 comp rx=0 tx=0).stop 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d88153f90 msgr2=0x7f8d88154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d88153f90 0x7f8d88154410 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8d80002240 tx=0x7f8d80009750 comp rx=0 tx=0).stop 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 shutdown_connections 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d680678f0 0x7f8d68069db0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d88153f90 0x7f8d88154410 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d8814b750 0x7f8d8814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 --2- 172.21.15.33:0/3112553446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8d8814b370 0x7f8d8815ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 >> 172.21.15.33:0/3112553446 conn(0x7f8d880b50c0 msgr2=0x7f8d8806eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:22.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 shutdown_connections 2024-04-25T01:09:22.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:22.708+0000 7f8d8df3c640 1 -- 172.21.15.33:0/3112553446 wait complete. 2024-04-25T01:09:22.715 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-04-25T01:09:22.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:22.716 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T01:09:22.818 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:22.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:09:22.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:22.819 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T01:09:22.830 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:09:22.830 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:22.837 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:09:22.837 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:22.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.068+0000 7f449e1f1640 1 Processor -- start 2024-04-25T01:09:23.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.068+0000 7f449e1f1640 1 -- start start 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f449e1f1640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f449806e1c0 0x7f449815e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f449e1f1640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f449815ea20 0x7f44980b4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f449e1f1640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4498147a20 con 0x7f4498147640 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f449e1f1640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f44981627e0 con 0x7f449806e1c0 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f449e1f1640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f449815f230 con 0x7f449815ea20 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f4496ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f449806e1c0 0x7f449815e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f44977fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f4498147640 0x7f449806dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49096/0 (socket says 172.21.15.33:49096) 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f4496ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f449806e1c0 0x7f449815e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60234/0 (socket says 172.21.15.33:60234) 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f44977fe640 1 -- 172.21.15.33:0/425657066 learned_addr learned my addr 172.21.15.33:0/425657066 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f4496ffd640 1 -- 172.21.15.33:0/425657066 >> v2:172.21.15.138:3300/0 conn(0x7f449815ea20 msgr2=0x7f44980b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f4496ffd640 1 --2- 172.21.15.33:0/425657066 >> v2:172.21.15.138:3300/0 conn(0x7f449815ea20 0x7f44980b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f4496ffd640 1 -- 172.21.15.33:0/425657066 >> v1:172.21.15.33:6789/0 conn(0x7f4498147640 legacy=0x7f449806dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f4496ffd640 1 -- 172.21.15.33:0/425657066 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4498161a20 con 0x7f449806e1c0 2024-04-25T01:09:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.072+0000 7f4496ffd640 1 --2- 172.21.15.33:0/425657066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815e4e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f44980af100 tx=0x7f448c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7643c0185ae3d223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44967fc640 1 -- 172.21.15.33:0/425657066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c0b1070 con 0x7f449806e1c0 2024-04-25T01:09:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44967fc640 1 -- 172.21.15.33:0/425657066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f448c0ac620 con 0x7f449806e1c0 2024-04-25T01:09:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44967fc640 1 -- 172.21.15.33:0/425657066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c0ac9f0 con 0x7f449806e1c0 2024-04-25T01:09:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- 172.21.15.33:0/425657066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 msgr2=0x7f449815e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 --2- 172.21.15.33:0/425657066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815e4e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f44980af100 tx=0x7f448c01c3c0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- 172.21.15.33:0/425657066 shutdown_connections 2024-04-25T01:09:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 --2- 172.21.15.33:0/425657066 >> v2:172.21.15.138:3300/0 conn(0x7f449815ea20 0x7f44980b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 --2- 172.21.15.33:0/425657066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815e4e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- 172.21.15.33:0/425657066 >> 172.21.15.33:0/425657066 conn(0x7f44980b1810 msgr2=0x7f44980b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- 172.21.15.33:0/425657066 shutdown_connections 2024-04-25T01:09:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- 172.21.15.33:0/425657066 wait complete. 2024-04-25T01:09:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 Processor -- start 2024-04-25T01:09:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- start start 2024-04-25T01:09:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4498147640 0x7f449813f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f449815ea20 0x7f449815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f449815eff0 con 0x7f449806e1c0 2024-04-25T01:09:23.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f449815ee70 con 0x7f4498147640 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f44980b5590 con 0x7f449815ea20 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60246/0 (socket says 172.21.15.33:60246) 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 -- 172.21.15.33:0/2599067831 learned_addr learned my addr 172.21.15.33:0/2599067831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 -- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f449815ea20 msgr2=0x7f449815be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f4496ffd640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4498147640 0x7f449813f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f4497fff640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f449815ea20 0x7f449815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f449815ea20 0x7f449815be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 -- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4498147640 msgr2=0x7f449813f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4498147640 0x7f449813f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 -- 172.21.15.33:0/2599067831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449815fa10 con 0x7f449806e1c0 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f4496ffd640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4498147640 0x7f449813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f44977fe640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815f430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f448800c420 tx=0x7f448800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f447ffff640 1 -- 172.21.15.33:0/2599067831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488017070 con 0x7f449806e1c0 2024-04-25T01:09:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f447ffff640 1 -- 172.21.15.33:0/2599067831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4488005cf0 con 0x7f449806e1c0 2024-04-25T01:09:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449815fcf0 con 0x7f449806e1c0 2024-04-25T01:09:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f447ffff640 1 -- 172.21.15.33:0/2599067831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44880072f0 con 0x7f449806e1c0 2024-04-25T01:09:23.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.076+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44981602b0 con 0x7f449806e1c0 2024-04-25T01:09:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.080+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44981606e0 con 0x7f449806e1c0 2024-04-25T01:09:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.080+0000 7f447ffff640 1 -- 172.21.15.33:0/2599067831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f448801a070 con 0x7f449806e1c0 2024-04-25T01:09:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.080+0000 7f447ffff640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f44840678c0 0x7f4484069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.080+0000 7f447ffff640 1 -- 172.21.15.33:0/2599067831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f44880854b0 con 0x7f449806e1c0 2024-04-25T01:09:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.080+0000 7f4496ffd640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f44840678c0 0x7f4484069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.080+0000 7f4496ffd640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f44840678c0 0x7f4484069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f448c0ae000 tx=0x7f448c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.084+0000 7f447ffff640 1 -- 172.21.15.33:0/2599067831 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f4488055bd0 con 0x7f449806e1c0 2024-04-25T01:09:23.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.248+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4498069040 con 0x7f449806e1c0 2024-04-25T01:09:23.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.248+0000 7f447ffff640 1 -- 172.21.15.33:0/2599067831 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7f44880554d0 con 0x7f449806e1c0 2024-04-25T01:09:23.242 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:23.243 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f44840678c0 msgr2=0x7f4484069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f44840678c0 0x7f4484069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f448c0ae000 tx=0x7f448c008040 comp rx=0 tx=0).stop 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 msgr2=0x7f449815f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815f430 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f448800c420 tx=0x7f448800c8e0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 shutdown_connections 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f44840678c0 0x7f4484069d80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f449815ea20 0x7f449815be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4498147640 0x7f449813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 --2- 172.21.15.33:0/2599067831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f449806e1c0 0x7f449815f430 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 >> 172.21.15.33:0/2599067831 conn(0x7f44980b1810 msgr2=0x7f4498070c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 shutdown_connections 2024-04-25T01:09:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.252+0000 7f449e1f1640 1 -- 172.21.15.33:0/2599067831 wait complete. 2024-04-25T01:09:23.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:23.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 Processor -- start 2024-04-25T01:09:23.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 -- start start 2024-04-25T01:09:23.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85e815ae20 0x7f85e80b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e8072410 0x7f85e80b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f85e806e5b0 0x7f85e806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f85e80b4a20 con 0x7f85e815ae20 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f85e80b2bb0 con 0x7f85e806e5b0 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85ee753640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85e80b2e90 con 0x7f85e8072410 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85e815ae20 0x7f85e80b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f85e815ae20 0x7f85e80b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60256/0 (socket says 172.21.15.33:60256) 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 -- 172.21.15.33:0/129988585 learned_addr learned my addr 172.21.15.33:0/129988585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e77fe640 1 --2- 172.21.15.33:0/129988585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e8072410 0x7f85e80b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 -- 172.21.15.33:0/129988585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e8072410 msgr2=0x7f85e80b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85eccf0640 1 --2- 172.21.15.33:0/129988585 >> v2:172.21.15.138:3300/0 conn(0x7f85e806e5b0 0x7f85e806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 --2- 172.21.15.33:0/129988585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e8072410 0x7f85e80b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 -- 172.21.15.33:0/129988585 >> v2:172.21.15.138:3300/0 conn(0x7f85e806e5b0 msgr2=0x7f85e806ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:23.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 --2- 172.21.15.33:0/129988585 >> v2:172.21.15.138:3300/0 conn(0x7f85e806e5b0 0x7f85e806ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e7fff640 1 -- 172.21.15.33:0/129988585 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e80b3170 con 0x7f85e815ae20 2024-04-25T01:09:23.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85e77fe640 1 --2- 172.21.15.33:0/129988585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e8072410 0x7f85e80b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:23.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.404+0000 7f85eccf0640 1 --2- 172.21.15.33:0/129988585 >> v2:172.21.15.138:3300/0 conn(0x7f85e806e5b0 0x7f85e806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:23.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e7fff640 1 --2- 172.21.15.33:0/129988585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e80b3690 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85e8073630 tx=0x7f85d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5686d1d10057f8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e6ffd640 1 -- 172.21.15.33:0/129988585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80ab5f0 con 0x7f85e815ae20 2024-04-25T01:09:23.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e6ffd640 1 -- 172.21.15.33:0/129988585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d80b0070 con 0x7f85e815ae20 2024-04-25T01:09:23.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e6ffd640 1 -- 172.21.15.33:0/129988585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d80ab9b0 con 0x7f85e815ae20 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- 172.21.15.33:0/129988585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 msgr2=0x7f85e80b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 --2- 172.21.15.33:0/129988585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e80b3690 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85e8073630 tx=0x7f85d801c520 comp rx=0 tx=0).stop 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- 172.21.15.33:0/129988585 shutdown_connections 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 --2- 172.21.15.33:0/129988585 >> v2:172.21.15.138:3300/0 conn(0x7f85e806e5b0 0x7f85e806ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 --2- 172.21.15.33:0/129988585 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e8072410 0x7f85e80b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 --2- 172.21.15.33:0/129988585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e80b3690 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- 172.21.15.33:0/129988585 >> 172.21.15.33:0/129988585 conn(0x7f85e80b1810 msgr2=0x7f85e80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- 172.21.15.33:0/129988585 shutdown_connections 2024-04-25T01:09:23.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- 172.21.15.33:0/129988585 wait complete. 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 Processor -- start 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- start start 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e806e5b0 0x7f85e8150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85e8072410 0x7f85e814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85e815f400 con 0x7f85e815ae20 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85e815f280 con 0x7f85e8072410 2024-04-25T01:09:23.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85e815f580 con 0x7f85e806e5b0 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e815dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60270/0 (socket says 172.21.15.33:60270) 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 -- 172.21.15.33:0/1825690878 learned_addr learned my addr 172.21.15.33:0/1825690878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e77fe640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85e8072410 0x7f85e814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e7fff640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e806e5b0 0x7f85e8150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 -- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e806e5b0 msgr2=0x7f85e8150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e806e5b0 0x7f85e8150260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 -- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85e8072410 msgr2=0x7f85e814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85e8072410 0x7f85e814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 -- 172.21.15.33:0/1825690878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e815e130 con 0x7f85e815ae20 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e77fe640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85e8072410 0x7f85e814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85eccf0640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e815dbf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f85e000cef0 tx=0x7f85e000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e4ff9640 1 -- 172.21.15.33:0/1825690878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e001a070 con 0x7f85e815ae20 2024-04-25T01:09:23.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e4ff9640 1 -- 172.21.15.33:0/1825690878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e000d840 con 0x7f85e815ae20 2024-04-25T01:09:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e815b430 con 0x7f85e815ae20 2024-04-25T01:09:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.408+0000 7f85e4ff9640 1 -- 172.21.15.33:0/1825690878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e000dc30 con 0x7f85e815ae20 2024-04-25T01:09:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.412+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e8156030 con 0x7f85e815ae20 2024-04-25T01:09:23.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.412+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e8155e50 con 0x7f85e815ae20 2024-04-25T01:09:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.412+0000 7f85e4ff9640 1 -- 172.21.15.33:0/1825690878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f85e001d070 con 0x7f85e815ae20 2024-04-25T01:09:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.412+0000 7f85e4ff9640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85d40678c0 0x7f85d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.412+0000 7f85e4ff9640 1 -- 172.21.15.33:0/1825690878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f85e0086960 con 0x7f85e815ae20 2024-04-25T01:09:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.412+0000 7f85e7fff640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85d40678c0 0x7f85d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.412+0000 7f85e7fff640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85d40678c0 0x7f85d4069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f85d80af000 tx=0x7f85d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.416+0000 7f85e4ff9640 1 -- 172.21.15.33:0/1825690878 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f85e8155e50 con 0x7f85e815ae20 2024-04-25T01:09:23.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.580+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85e8069040 con 0x7f85e815ae20 2024-04-25T01:09:23.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.580+0000 7f85e4ff9640 1 -- 172.21.15.33:0/1825690878 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f85e0057000 con 0x7f85e815ae20 2024-04-25T01:09:23.575 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85d40678c0 msgr2=0x7f85d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85d40678c0 0x7f85d4069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f85d80af000 tx=0x7f85d8008040 comp rx=0 tx=0).stop 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 msgr2=0x7f85e815dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e815dbf0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f85e000cef0 tx=0x7f85e000a710 comp rx=0 tx=0).stop 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 shutdown_connections 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85d40678c0 0x7f85d4069d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85e815ae20 0x7f85e815dbf0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85e8072410 0x7f85e814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 --2- 172.21.15.33:0/1825690878 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85e806e5b0 0x7f85e8150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 >> 172.21.15.33:0/1825690878 conn(0x7f85e80b1810 msgr2=0x7f85e81533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 shutdown_connections 2024-04-25T01:09:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.584+0000 7f85ee753640 1 -- 172.21.15.33:0/1825690878 wait complete. 2024-04-25T01:09:23.594 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:23.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bc14e5640 1 Processor -- start 2024-04-25T01:09:23.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bc14e5640 1 -- start start 2024-04-25T01:09:23.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bc14e5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6bbc145660 0x7f6bbc06eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bc14e5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 0x7f6bbc071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bc14e5640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bbc0cee10 con 0x7f6bbc15f960 2024-04-25T01:09:23.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bc14e5640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6bbc06f0a0 con 0x7f6bbc145660 2024-04-25T01:09:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bc14e5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc145a40 con 0x7f6bbc1604b0 2024-04-25T01:09:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbaffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6bbc145660 0x7f6bbc06eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbaffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6bbc145660 0x7f6bbc06eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60286/0 (socket says 172.21.15.33:60286) 2024-04-25T01:09:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbaffd640 1 -- 172.21.15.33:0/3269797291 learned_addr learned my addr 172.21.15.33:0/3269797291 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbb7fe640 1 --2- 172.21.15.33:0/3269797291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 0x7f6bbc071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbaffd640 1 -- 172.21.15.33:0/3269797291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 msgr2=0x7f6bbc071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:23.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbaffd640 1 --2- 172.21.15.33:0/3269797291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 0x7f6bbc071170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbaffd640 1 -- 172.21.15.33:0/3269797291 >> v1:172.21.15.138:6789/0 conn(0x7f6bbc15f960 legacy=0x7f6bbc15fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:23.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.720+0000 7f6bbaffd640 1 -- 172.21.15.33:0/3269797291 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc0cf0f0 con 0x7f6bbc145660 2024-04-25T01:09:23.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bbaffd640 1 --2- 172.21.15.33:0/3269797291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc145660 0x7f6bbc06eb60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009870 tx=0x7f6bac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fdfa828a9c7b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bb9ffb640 1 -- 172.21.15.33:0/3269797291 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0b1070 con 0x7f6bbc145660 2024-04-25T01:09:23.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bb9ffb640 1 -- 172.21.15.33:0/3269797291 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac0ac630 con 0x7f6bbc145660 2024-04-25T01:09:23.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bb9ffb640 1 -- 172.21.15.33:0/3269797291 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0a8050 con 0x7f6bbc145660 2024-04-25T01:09:23.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 -- 172.21.15.33:0/3269797291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc145660 msgr2=0x7f6bbc06eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/3269797291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc145660 0x7f6bbc06eb60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009870 tx=0x7f6bac01c520 comp rx=0 tx=0).stop 2024-04-25T01:09:23.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 -- 172.21.15.33:0/3269797291 shutdown_connections 2024-04-25T01:09:23.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/3269797291 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 0x7f6bbc071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/3269797291 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc145660 0x7f6bbc06eb60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 -- 172.21.15.33:0/3269797291 >> 172.21.15.33:0/3269797291 conn(0x7f6bbc0b1810 msgr2=0x7f6bbc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:23.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 -- 172.21.15.33:0/3269797291 shutdown_connections 2024-04-25T01:09:23.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 -- 172.21.15.33:0/3269797291 wait complete. 2024-04-25T01:09:23.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 Processor -- start 2024-04-25T01:09:23.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.724+0000 7f6bc14e5640 1 -- start start 2024-04-25T01:09:23.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc15f960 0x7f6bbc158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 0x7f6bbc0b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bbc0b3030 0x7f6bbc142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc072290 con 0x7f6bbc15f960 2024-04-25T01:09:23.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bbc072110 con 0x7f6bbc0b3030 2024-04-25T01:09:23.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bbc072410 con 0x7f6bbc1604b0 2024-04-25T01:09:23.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc15f960 0x7f6bbc158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc15f960 0x7f6bbc158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60302/0 (socket says 172.21.15.33:60302) 2024-04-25T01:09:23.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 -- 172.21.15.33:0/573041584 learned_addr learned my addr 172.21.15.33:0/573041584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:23.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 -- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 msgr2=0x7f6bbc0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 0x7f6bbc0b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 -- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bbc0b3030 msgr2=0x7f6bbc142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bbc0b3030 0x7f6bbc142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 -- 172.21.15.33:0/573041584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bbc143c80 con 0x7f6bbc15f960 2024-04-25T01:09:23.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bbaffd640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc15f960 0x7f6bbc158540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0af040 tx=0x7f6bac0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6ba37fe640 1 -- 172.21.15.33:0/573041584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0bc070 con 0x7f6bbc15f960 2024-04-25T01:09:23.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6ba37fe640 1 -- 172.21.15.33:0/573041584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bac0b7630 con 0x7f6bbc15f960 2024-04-25T01:09:23.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bbc143f60 con 0x7f6bbc15f960 2024-04-25T01:09:23.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6ba37fe640 1 -- 172.21.15.33:0/573041584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac0b1040 con 0x7f6bbc15f960 2024-04-25T01:09:23.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bbc144520 con 0x7f6bbc15f960 2024-04-25T01:09:23.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bbc069040 con 0x7f6bbc15f960 2024-04-25T01:09:23.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6ba37fe640 1 -- 172.21.15.33:0/573041584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6bac0bf070 con 0x7f6bbc15f960 2024-04-25T01:09:23.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6ba37fe640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6ba80678c0 0x7f6ba8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:23.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6ba37fe640 1 -- 172.21.15.33:0/573041584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6bac11f7e0 con 0x7f6bbc15f960 2024-04-25T01:09:23.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.728+0000 7f6bba7fc640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6ba80678c0 0x7f6ba8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:23.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.732+0000 7f6bba7fc640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6ba80678c0 0x7f6ba8069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc133d00 tx=0x7f6bb0006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:23.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.732+0000 7f6ba37fe640 1 -- 172.21.15.33:0/573041584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6bac016110 con 0x7f6bbc15f960 2024-04-25T01:09:23.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.888+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6bbc072ac0 con 0x7f6bbc15f960 2024-04-25T01:09:23.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.888+0000 7f6ba37fe640 1 -- 172.21.15.33:0/573041584 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f6bac0eff20 con 0x7f6bbc15f960 2024-04-25T01:09:23.883 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6ba80678c0 msgr2=0x7f6ba8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6ba80678c0 0x7f6ba8069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc133d00 tx=0x7f6bb0006bd0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc15f960 msgr2=0x7f6bbc158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc15f960 0x7f6bbc158540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6bac0af040 tx=0x7f6bac0192c0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 shutdown_connections 2024-04-25T01:09:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6ba80678c0 0x7f6ba8069d80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bbc0b3030 0x7f6bbc142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bbc1604b0 0x7f6bbc0b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 --2- 172.21.15.33:0/573041584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bbc15f960 0x7f6bbc158540 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 >> 172.21.15.33:0/573041584 conn(0x7f6bbc0b1810 msgr2=0x7f6bbc15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:23.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 shutdown_connections 2024-04-25T01:09:23.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:23.892+0000 7f6bc14e5640 1 -- 172.21.15.33:0/573041584 wait complete. 2024-04-25T01:09:23.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:24.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1ce0848640 1 Processor -- start 2024-04-25T01:09:24.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1ce0848640 1 -- start start 2024-04-25T01:09:24.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1ce0848640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1cdc06eb60 0x7f1cdc160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1ce0848640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc071170 0x7f1cdc071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1ce0848640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cdc0cee10 con 0x7f1cdc145660 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1ce0848640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1cdc145a40 con 0x7f1cdc06eb60 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1ce0848640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc070ce0 con 0x7f1cdc071170 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cd3fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1cdc06eb60 0x7f1cdc160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cd3fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1cdc06eb60 0x7f1cdc160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60314/0 (socket says 172.21.15.33:60314) 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cd3fff640 1 -- 172.21.15.33:0/1393196129 learned_addr learned my addr 172.21.15.33:0/1393196129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:24.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cdad9d640 1 --2- 172.21.15.33:0/1393196129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc071170 0x7f1cdc071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cd3fff640 1 -- 172.21.15.33:0/1393196129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc071170 msgr2=0x7f1cdc071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cd3fff640 1 --2- 172.21.15.33:0/1393196129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc071170 0x7f1cdc071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cd3fff640 1 -- 172.21.15.33:0/1393196129 >> v1:172.21.15.138:6789/0 conn(0x7f1cdc145660 legacy=0x7f1cdc15f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:24.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.032+0000 7f1cd3fff640 1 -- 172.21.15.33:0/1393196129 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc0cf0f0 con 0x7f1cdc06eb60 2024-04-25T01:09:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1cd3fff640 1 --2- 172.21.15.33:0/1393196129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc06eb60 0x7f1cdc160070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009870 tx=0x7f1cc401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cf1c437556750c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1cd9d9b640 1 -- 172.21.15.33:0/1393196129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40aa610 con 0x7f1cdc06eb60 2024-04-25T01:09:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1cd9d9b640 1 -- 172.21.15.33:0/1393196129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc40af070 con 0x7f1cdc06eb60 2024-04-25T01:09:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1cd9d9b640 1 -- 172.21.15.33:0/1393196129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40aa9d0 con 0x7f1cdc06eb60 2024-04-25T01:09:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- 172.21.15.33:0/1393196129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc06eb60 msgr2=0x7f1cdc160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 --2- 172.21.15.33:0/1393196129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc06eb60 0x7f1cdc160070 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f1cc4009870 tx=0x7f1cc401c4f0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- 172.21.15.33:0/1393196129 shutdown_connections 2024-04-25T01:09:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 --2- 172.21.15.33:0/1393196129 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc071170 0x7f1cdc071600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 --2- 172.21.15.33:0/1393196129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc06eb60 0x7f1cdc160070 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- 172.21.15.33:0/1393196129 >> 172.21.15.33:0/1393196129 conn(0x7f1cdc0b1810 msgr2=0x7f1cdc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- 172.21.15.33:0/1393196129 shutdown_connections 2024-04-25T01:09:24.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- 172.21.15.33:0/1393196129 wait complete. 2024-04-25T01:09:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 Processor -- start 2024-04-25T01:09:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- start start 2024-04-25T01:09:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc06eb60 0x7f1cdc136080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc071170 0x7f1cdc137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc145660 0x7f1cdc15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc160730 con 0x7f1cdc071170 2024-04-25T01:09:24.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cdc1605b0 con 0x7f1cdc145660 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1ce0848640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1cdc168ca0 con 0x7f1cdc06eb60 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1cd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc071170 0x7f1cdc137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1cd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc071170 0x7f1cdc137a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60330/0 (socket says 172.21.15.33:60330) 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.036+0000 7f1cd3fff640 1 -- 172.21.15.33:0/196038371 learned_addr learned my addr 172.21.15.33:0/196038371 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cda59c640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc06eb60 0x7f1cdc136080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cdad9d640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc145660 0x7f1cdc15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd3fff640 1 -- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc06eb60 msgr2=0x7f1cdc136080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd3fff640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc06eb60 0x7f1cdc136080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd3fff640 1 -- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc145660 msgr2=0x7f1cdc15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd3fff640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc145660 0x7f1cdc15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd3fff640 1 -- 172.21.15.33:0/196038371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cdc137f80 con 0x7f1cdc071170 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd3fff640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc071170 0x7f1cdc137a40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40ac040 tx=0x7f1cc401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd2ffd640 1 -- 172.21.15.33:0/196038371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40bd5f0 con 0x7f1cdc071170 2024-04-25T01:09:24.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd2ffd640 1 -- 172.21.15.33:0/196038371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc40bd7b0 con 0x7f1cdc071170 2024-04-25T01:09:24.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cdc138240 con 0x7f1cdc071170 2024-04-25T01:09:24.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd2ffd640 1 -- 172.21.15.33:0/196038371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40bdba0 con 0x7f1cdc071170 2024-04-25T01:09:24.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cdc13e1b0 con 0x7f1cdc071170 2024-04-25T01:09:24.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cdc069040 con 0x7f1cdc071170 2024-04-25T01:09:24.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd2ffd640 1 -- 172.21.15.33:0/196038371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1cdc072350 con 0x7f1cdc071170 2024-04-25T01:09:24.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd2ffd640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cbc0678c0 0x7f1cbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cd2ffd640 1 -- 172.21.15.33:0/196038371 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1cc4120af0 con 0x7f1cdc071170 2024-04-25T01:09:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cda59c640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cbc0678c0 0x7f1cbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.040+0000 7f1cda59c640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cbc0678c0 0x7f1cbc069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00b810 tx=0x7f1ccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.044+0000 7f1cd2ffd640 1 -- 172.21.15.33:0/196038371 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1cc40c3060 con 0x7f1cdc071170 2024-04-25T01:09:24.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.204+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1cdc069040 con 0x7f1cdc071170 2024-04-25T01:09:24.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.204+0000 7f1cd2ffd640 1 -- 172.21.15.33:0/196038371 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f1cc40a7070 con 0x7f1cdc071170 2024-04-25T01:09:24.198 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cbc0678c0 msgr2=0x7f1cbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cbc0678c0 0x7f1cbc069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00b810 tx=0x7f1ccc008040 comp rx=0 tx=0).stop 2024-04-25T01:09:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc071170 msgr2=0x7f1cdc137a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc071170 0x7f1cdc137a40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f1cc40ac040 tx=0x7f1cc401ad30 comp rx=0 tx=0).stop 2024-04-25T01:09:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 shutdown_connections 2024-04-25T01:09:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1cbc0678c0 0x7f1cbc069d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cdc145660 0x7f1cdc15b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1cdc071170 0x7f1cdc137a40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 --2- 172.21.15.33:0/196038371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1cdc06eb60 0x7f1cdc136080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 >> 172.21.15.33:0/196038371 conn(0x7f1cdc0b1810 msgr2=0x7f1cdc13b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 shutdown_connections 2024-04-25T01:09:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.208+0000 7f1ce0848640 1 -- 172.21.15.33:0/196038371 wait complete. 2024-04-25T01:09:24.217 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:24.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21c872640 1 Processor -- start 2024-04-25T01:09:24.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21c872640 1 -- start start 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21c872640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff2180b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21c872640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff21815ae70 0x7ff21806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21c872640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2180b4840 con 0x7ff21806dfe0 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21c872640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff2180b5410 con 0x7ff21815ae70 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21c872640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180d0100 con 0x7ff2180cd580 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21659c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff2180b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff216d9d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff21815ae70 0x7ff21806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff215d9b640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7ff21806dfe0 0x7ff2180cd960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:53304/0 (socket says 172.21.15.33:53304) 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21659c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff2180b58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:41722/0 (socket says 172.21.15.33:41722) 2024-04-25T01:09:24.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff215d9b640 1 -- 172.21.15.33:0/3380739195 learned_addr learned my addr 172.21.15.33:0/3380739195 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:24.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21659c640 1 -- 172.21.15.33:0/3380739195 >> v2:172.21.15.138:3300/0 conn(0x7ff21815ae70 msgr2=0x7ff21806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:24.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21659c640 1 --2- 172.21.15.33:0/3380739195 >> v2:172.21.15.138:3300/0 conn(0x7ff21815ae70 0x7ff21806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21659c640 1 -- 172.21.15.33:0/3380739195 >> v1:172.21.15.138:6789/0 conn(0x7ff21806dfe0 legacy=0x7ff2180cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.352+0000 7ff21659c640 1 -- 172.21.15.33:0/3380739195 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2180b4ae0 con 0x7ff2180cd580 2024-04-25T01:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21659c640 1 --2- 172.21.15.33:0/3380739195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff2180b58a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff2000097f0 tx=0x7ff20001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c90bcac12649762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21559a640 1 -- 172.21.15.33:0/3380739195 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2000a7070 con 0x7ff2180cd580 2024-04-25T01:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21559a640 1 -- 172.21.15.33:0/3380739195 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2000b0070 con 0x7ff2180cd580 2024-04-25T01:09:24.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21559a640 1 -- 172.21.15.33:0/3380739195 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2000ab890 con 0x7ff2180cd580 2024-04-25T01:09:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 -- 172.21.15.33:0/3380739195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 msgr2=0x7ff2180b58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 --2- 172.21.15.33:0/3380739195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff2180b58a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff2000097f0 tx=0x7ff20001c470 comp rx=0 tx=0).stop 2024-04-25T01:09:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 -- 172.21.15.33:0/3380739195 shutdown_connections 2024-04-25T01:09:24.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 --2- 172.21.15.33:0/3380739195 >> v2:172.21.15.138:3300/0 conn(0x7ff21815ae70 0x7ff21806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 --2- 172.21.15.33:0/3380739195 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff2180b58a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 -- 172.21.15.33:0/3380739195 >> 172.21.15.33:0/3380739195 conn(0x7ff2180677a0 msgr2=0x7ff21806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 -- 172.21.15.33:0/3380739195 shutdown_connections 2024-04-25T01:09:24.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.356+0000 7ff21c872640 1 -- 172.21.15.33:0/3380739195 wait complete. 2024-04-25T01:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 Processor -- start 2024-04-25T01:09:24.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 -- start start 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff21806dfe0 0x7ff2181520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff218152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815ae70 0x7ff218166880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff218165870 con 0x7ff21815ae70 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2180cfea0 con 0x7ff21806dfe0 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2181659f0 con 0x7ff2180cd580 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815ae70 0x7ff218166880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815ae70 0x7ff218166880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60338/0 (socket says 172.21.15.33:60338) 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 -- 172.21.15.33:0/299142431 learned_addr learned my addr 172.21.15.33:0/299142431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff215d9b640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff218152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21659c640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff21806dfe0 0x7ff2181520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 -- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 msgr2=0x7ff218152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff218152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 -- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff21806dfe0 msgr2=0x7ff2181520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff21806dfe0 0x7ff2181520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 -- 172.21.15.33:0/299142431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21815cf20 con 0x7ff21815ae70 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff216d9d640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815ae70 0x7ff218166880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff21000e7c0 tx=0x7ff21000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff20effd640 1 -- 172.21.15.33:0/299142431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210017070 con 0x7ff21815ae70 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff20effd640 1 -- 172.21.15.33:0/299142431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff21000ee70 con 0x7ff21815ae70 2024-04-25T01:09:24.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff21815d200 con 0x7ff21815ae70 2024-04-25T01:09:24.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff20effd640 1 -- 172.21.15.33:0/299142431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000cb40 con 0x7ff21815ae70 2024-04-25T01:09:24.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff21814dd80 con 0x7ff21815ae70 2024-04-25T01:09:24.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.360+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff218069040 con 0x7ff21815ae70 2024-04-25T01:09:24.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.364+0000 7ff20effd640 1 -- 172.21.15.33:0/299142431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff21001a070 con 0x7ff21815ae70 2024-04-25T01:09:24.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.364+0000 7ff20effd640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1f80678c0 0x7ff1f8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.364+0000 7ff20effd640 1 -- 172.21.15.33:0/299142431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff210086220 con 0x7ff21815ae70 2024-04-25T01:09:24.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.364+0000 7ff21659c640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1f80678c0 0x7ff1f8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.364+0000 7ff21659c640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1f80678c0 0x7ff1f8069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2000af000 tx=0x7ff2000a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.368+0000 7ff20effd640 1 -- 172.21.15.33:0/299142431 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff210056970 con 0x7ff21815ae70 2024-04-25T01:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff218168870 con 0x7ff21815ae70 2024-04-25T01:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff20effd640 1 -- 172.21.15.33:0/299142431 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff210056270 con 0x7ff21815ae70 2024-04-25T01:09:24.525 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1f80678c0 msgr2=0x7ff1f8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1f80678c0 0x7ff1f8069d80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2000af000 tx=0x7ff2000a7000 comp rx=0 tx=0).stop 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815ae70 msgr2=0x7ff218166880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815ae70 0x7ff218166880 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff21000e7c0 tx=0x7ff21000ec80 comp rx=0 tx=0).stop 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 shutdown_connections 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1f80678c0 0x7ff1f8069d80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff21815ae70 0x7ff218166880 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2180cd580 0x7ff218152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 --2- 172.21.15.33:0/299142431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff21806dfe0 0x7ff2181520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.532+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 >> 172.21.15.33:0/299142431 conn(0x7ff2180677a0 msgr2=0x7ff2180b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.536+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 shutdown_connections 2024-04-25T01:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.536+0000 7ff21c872640 1 -- 172.21.15.33:0/299142431 wait complete. 2024-04-25T01:09:24.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb264386640 1 Processor -- start 2024-04-25T01:09:24.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb264386640 1 -- start start 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb264386640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb25c1444f0 0x7fb25c0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb264386640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb264386640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb25c1448d0 con 0x7fb25c072370 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb264386640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb25c0ce0c0 con 0x7fb25c1444f0 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb264386640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c0b5b60 con 0x7fb25c15e660 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb262122640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb25c1444f0 0x7fb25c0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb262122640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb25c1444f0 0x7fb25c0b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60344/0 (socket says 172.21.15.33:60344) 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb262122640 1 -- 172.21.15.33:0/1008112730 learned_addr learned my addr 172.21.15.33:0/1008112730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb261921640 1 --2- 172.21.15.33:0/1008112730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb262122640 1 -- 172.21.15.33:0/1008112730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 msgr2=0x7fb25c071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb262122640 1 --2- 172.21.15.33:0/1008112730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb262122640 1 -- 172.21.15.33:0/1008112730 >> v1:172.21.15.138:6789/0 conn(0x7fb25c072370 legacy=0x7fb25c06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:24.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.680+0000 7fb262122640 1 -- 172.21.15.33:0/1008112730 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c15ec20 con 0x7fb25c1444f0 2024-04-25T01:09:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb262122640 1 --2- 172.21.15.33:0/1008112730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c0b5620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb258009870 tx=0x7fb25801c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6556869091e3180 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261120640 1 -- 172.21.15.33:0/1008112730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2580b0070 con 0x7fb25c1444f0 2024-04-25T01:09:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261120640 1 -- 172.21.15.33:0/1008112730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2580ab600 con 0x7fb25c1444f0 2024-04-25T01:09:24.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261120640 1 -- 172.21.15.33:0/1008112730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2580ab9d0 con 0x7fb25c1444f0 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- 172.21.15.33:0/1008112730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 msgr2=0x7fb25c0b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 --2- 172.21.15.33:0/1008112730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c0b5620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb258009870 tx=0x7fb25801c2b0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- 172.21.15.33:0/1008112730 shutdown_connections 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 --2- 172.21.15.33:0/1008112730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 --2- 172.21.15.33:0/1008112730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c0b5620 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- 172.21.15.33:0/1008112730 >> 172.21.15.33:0/1008112730 conn(0x7fb25c0b1810 msgr2=0x7fb25c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- 172.21.15.33:0/1008112730 shutdown_connections 2024-04-25T01:09:24.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- 172.21.15.33:0/1008112730 wait complete. 2024-04-25T01:09:24.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 Processor -- start 2024-04-25T01:09:24.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- start start 2024-04-25T01:09:24.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25c072370 0x7fb25c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c0b37a0 con 0x7fb25c1444f0 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb25c0b3620 con 0x7fb25c072370 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb264386640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb25c167a00 con 0x7fb25c15e660 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c14e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60356/0 (socket says 172.21.15.33:60356) 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 -- 172.21.15.33:0/1250028828 learned_addr learned my addr 172.21.15.33:0/1250028828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb262122640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25c072370 0x7fb25c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb262923640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 -- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 msgr2=0x7fb25c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 -- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25c072370 msgr2=0x7fb25c0a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25c072370 0x7fb25c0a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb261921640 1 -- 172.21.15.33:0/1250028828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb25c0aa130 con 0x7fb25c1444f0 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.684+0000 7fb262923640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb261921640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c14e300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d5f0 tx=0x7fb24c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb252ffd640 1 -- 172.21.15.33:0/1250028828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0125f0 con 0x7fb25c1444f0 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb252ffd640 1 -- 172.21.15.33:0/1250028828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb24c019070 con 0x7fb25c1444f0 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25c0aa410 con 0x7fb25c1444f0 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb252ffd640 1 -- 172.21.15.33:0/1250028828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24c0129b0 con 0x7fb25c1444f0 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25c0aa9d0 con 0x7fb25c1444f0 2024-04-25T01:09:24.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25c069040 con 0x7fb25c1444f0 2024-04-25T01:09:24.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb252ffd640 1 -- 172.21.15.33:0/1250028828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb24c012b50 con 0x7fb25c1444f0 2024-04-25T01:09:24.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb252ffd640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb2400678c0 0x7fb240069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:24.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb252ffd640 1 -- 172.21.15.33:0/1250028828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb24c0864a0 con 0x7fb25c1444f0 2024-04-25T01:09:24.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb262122640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb2400678c0 0x7fb240069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:24.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.688+0000 7fb262122640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb2400678c0 0x7fb240069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2580ad000 tx=0x7fb2580a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:24.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.692+0000 7fb252ffd640 1 -- 172.21.15.33:0/1250028828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb24c056c70 con 0x7fb25c1444f0 2024-04-25T01:09:24.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb25c0b2f20 con 0x7fb25c1444f0 2024-04-25T01:09:24.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb252ffd640 1 -- 172.21.15.33:0/1250028828 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fb24c056570 con 0x7fb25c1444f0 2024-04-25T01:09:24.861 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:24.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb2400678c0 msgr2=0x7fb240069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb2400678c0 0x7fb240069d80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2580ad000 tx=0x7fb2580a6000 comp rx=0 tx=0).stop 2024-04-25T01:09:24.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 msgr2=0x7fb25c14e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c14e300 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d5f0 tx=0x7fb24c00dab0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 shutdown_connections 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb2400678c0 0x7fb240069d80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.868+0000 7fb264386640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb25c15e660 0x7fb25c071500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.872+0000 7fb264386640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb25c1444f0 0x7fb25c14e300 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.872+0000 7fb264386640 1 --2- 172.21.15.33:0/1250028828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb25c072370 0x7fb25c0a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.872+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 >> 172.21.15.33:0/1250028828 conn(0x7fb25c0b1810 msgr2=0x7fb25c13b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.872+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 shutdown_connections 2024-04-25T01:09:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:24.872+0000 7fb264386640 1 -- 172.21.15.33:0/1250028828 wait complete. 2024-04-25T01:09:24.879 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:09:24.879 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:24.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:25.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.048+0000 7efd4380d640 1 Processor -- start 2024-04-25T01:09:25.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.048+0000 7efd4380d640 1 -- start start 2024-04-25T01:09:25.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd3c0b52d0 0x7efd3c0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3c15b490 con 0x7efd3c06c3d0 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd3c15b770 con 0x7efd3c151b10 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7efd3c15ba50 con 0x7efd3c0b52d0 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd40da8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd3c0b52d0 0x7efd3c0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd40da8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efd3c0b52d0 0x7efd3c0cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60368/0 (socket says 172.21.15.33:60368) 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd40da8640 1 -- 172.21.15.33:0/2138419886 learned_addr learned my addr 172.21.15.33:0/2138419886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd41daa640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efd3c06c3d0 0x7efd3c0cf200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49104/0 (socket says 172.21.15.33:49104) 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd40da8640 1 -- 172.21.15.33:0/2138419886 >> v1:172.21.15.138:6789/0 conn(0x7efd3c151b10 legacy=0x7efd3c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd40da8640 1 -- 172.21.15.33:0/2138419886 >> v1:172.21.15.33:6789/0 conn(0x7efd3c06c3d0 legacy=0x7efd3c0cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd40da8640 1 -- 172.21.15.33:0/2138419886 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c15bd30 con 0x7efd3c0b52d0 2024-04-25T01:09:25.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd40da8640 1 --2- 172.21.15.33:0/2138419886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c0cecc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efd30009870 tx=0x7efd3001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baf6cc1c55e1ab9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd3bfff640 1 -- 172.21.15.33:0/2138419886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300aa610 con 0x7efd3c0b52d0 2024-04-25T01:09:25.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd3bfff640 1 -- 172.21.15.33:0/2138419886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd300af070 con 0x7efd3c0b52d0 2024-04-25T01:09:25.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd3bfff640 1 -- 172.21.15.33:0/2138419886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300aa9d0 con 0x7efd3c0b52d0 2024-04-25T01:09:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- 172.21.15.33:0/2138419886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 msgr2=0x7efd3c0cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 --2- 172.21.15.33:0/2138419886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c0cecc0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7efd30009870 tx=0x7efd3001c520 comp rx=0 tx=0).stop 2024-04-25T01:09:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- 172.21.15.33:0/2138419886 shutdown_connections 2024-04-25T01:09:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 --2- 172.21.15.33:0/2138419886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c0cecc0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- 172.21.15.33:0/2138419886 >> 172.21.15.33:0/2138419886 conn(0x7efd3c0b2340 msgr2=0x7efd3c0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:25.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- 172.21.15.33:0/2138419886 shutdown_connections 2024-04-25T01:09:25.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.052+0000 7efd4380d640 1 -- 172.21.15.33:0/2138419886 wait complete. 2024-04-25T01:09:25.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 Processor -- start 2024-04-25T01:09:25.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 -- start start 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06c3d0 0x7efd3c140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c151b10 0x7efd3c1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c15b080 con 0x7efd3c0b52d0 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c0cfd20 con 0x7efd3c151b10 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c15b200 con 0x7efd3c06c3d0 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60380/0 (socket says 172.21.15.33:60380) 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 -- 172.21.15.33:0/214656627 learned_addr learned my addr 172.21.15.33:0/214656627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 -- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06c3d0 msgr2=0x7efd3c140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd415a9640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06c3d0 0x7efd3c140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd41daa640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c151b10 0x7efd3c1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06c3d0 0x7efd3c140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 -- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c151b10 msgr2=0x7efd3c1506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c151b10 0x7efd3c1506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 -- 172.21.15.33:0/214656627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c142c10 con 0x7efd3c0b52d0 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd40da8640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c154aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efd300ac040 tx=0x7efd3001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd39ffb640 1 -- 172.21.15.33:0/214656627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300b3800 con 0x7efd3c0b52d0 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd39ffb640 1 -- 172.21.15.33:0/214656627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd300b39c0 con 0x7efd3c0b52d0 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd39ffb640 1 -- 172.21.15.33:0/214656627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd300b3db0 con 0x7efd3c0b52d0 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c141e60 con 0x7efd3c0b52d0 2024-04-25T01:09:25.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c142420 con 0x7efd3c0b52d0 2024-04-25T01:09:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.056+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c069040 con 0x7efd3c0b52d0 2024-04-25T01:09:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.060+0000 7efd39ffb640 1 -- 172.21.15.33:0/214656627 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efd300b2380 con 0x7efd3c0b52d0 2024-04-25T01:09:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.060+0000 7efd39ffb640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efd280678c0 0x7efd28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.060+0000 7efd39ffb640 1 -- 172.21.15.33:0/214656627 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efd30120230 con 0x7efd3c0b52d0 2024-04-25T01:09:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.060+0000 7efd415a9640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efd280678c0 0x7efd28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.060+0000 7efd415a9640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efd280678c0 0x7efd28069d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efd3c153350 tx=0x7efd34007310 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.064+0000 7efd39ffb640 1 -- 172.21.15.33:0/214656627 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efd3c142c10 con 0x7efd3c0b52d0 2024-04-25T01:09:25.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.224+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd3c146620 con 0x7efd3c0b52d0 2024-04-25T01:09:25.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.224+0000 7efd39ffb640 1 -- 172.21.15.33:0/214656627 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7efd300a7070 con 0x7efd3c0b52d0 2024-04-25T01:09:25.218 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:25.218 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efd280678c0 msgr2=0x7efd28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efd280678c0 0x7efd28069d80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7efd3c153350 tx=0x7efd34007310 comp rx=0 tx=0).stop 2024-04-25T01:09:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 msgr2=0x7efd3c154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c154aa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efd300ac040 tx=0x7efd3001ad60 comp rx=0 tx=0).stop 2024-04-25T01:09:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 shutdown_connections 2024-04-25T01:09:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efd280678c0 0x7efd28069d80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efd3c151b10 0x7efd3c1506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd3c0b52d0 0x7efd3c154aa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 --2- 172.21.15.33:0/214656627 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efd3c06c3d0 0x7efd3c140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 >> 172.21.15.33:0/214656627 conn(0x7efd3c0b2340 msgr2=0x7efd3c0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 shutdown_connections 2024-04-25T01:09:25.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.228+0000 7efd4380d640 1 -- 172.21.15.33:0/214656627 wait complete. 2024-04-25T01:09:25.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:25.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb81b35e640 1 Processor -- start 2024-04-25T01:09:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb81b35e640 1 -- start start 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb81b35e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb81406b210 0x7fb81415e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb81b35e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8140ae170 con 0x7fb814071df0 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb81b35e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb814072640 con 0x7fb8140add10 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb81b35e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb8140ceb80 con 0x7fb81406b210 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb8198fb640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb814071df0 0x7fb8140cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49112/0 (socket says 172.21.15.33:49112) 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb8198fb640 1 -- 172.21.15.33:0/3148988084 learned_addr learned my addr 172.21.15.33:0/3148988084 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb8190fa640 1 --2- 172.21.15.33:0/3148988084 >> v2:172.21.15.33:3300/0 conn(0x7fb81406b210 0x7fb81415e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb80bfff640 1 -- 172.21.15.33:0/3148988084 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367517606 0 0) 0x7fb80c0014a0 con 0x7fb814071df0 2024-04-25T01:09:25.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb80bfff640 1 -- 172.21.15.33:0/3148988084 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb800003530 con 0x7fb814071df0 2024-04-25T01:09:25.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb8190fa640 1 -- 172.21.15.33:0/3148988084 >> v1:172.21.15.138:6789/0 conn(0x7fb8140add10 legacy=0x7fb81406b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb8190fa640 1 -- 172.21.15.33:0/3148988084 >> v1:172.21.15.33:6789/0 conn(0x7fb814071df0 legacy=0x7fb8140cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.368+0000 7fb8190fa640 1 -- 172.21.15.33:0/3148988084 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb814072920 con 0x7fb81406b210 2024-04-25T01:09:25.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb8190fa640 1 --2- 172.21.15.33:0/3148988084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb81406b210 0x7fb81415e6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb8040097c0 tx=0x7fb80401ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2f89ddf94de684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb80bfff640 1 -- 172.21.15.33:0/3148988084 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040ab600 con 0x7fb81406b210 2024-04-25T01:09:25.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb80bfff640 1 -- 172.21.15.33:0/3148988084 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8040b0070 con 0x7fb81406b210 2024-04-25T01:09:25.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb80bfff640 1 -- 172.21.15.33:0/3148988084 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8040ab9c0 con 0x7fb81406b210 2024-04-25T01:09:25.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- 172.21.15.33:0/3148988084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb81406b210 msgr2=0x7fb81415e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3148988084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb81406b210 0x7fb81415e6f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb8040097c0 tx=0x7fb80401ce40 comp rx=0 tx=0).stop 2024-04-25T01:09:25.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- 172.21.15.33:0/3148988084 shutdown_connections 2024-04-25T01:09:25.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3148988084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb81406b210 0x7fb81415e6f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- 172.21.15.33:0/3148988084 >> 172.21.15.33:0/3148988084 conn(0x7fb8140b28c0 msgr2=0x7fb8140b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:25.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- 172.21.15.33:0/3148988084 shutdown_connections 2024-04-25T01:09:25.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- 172.21.15.33:0/3148988084 wait complete. 2024-04-25T01:09:25.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 Processor -- start 2024-04-25T01:09:25.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- start start 2024-04-25T01:09:25.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb81406b210 0x7fb814156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb814071df0 0x7fb8141580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8140add10 0x7fb81415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb814072380 con 0x7fb8140add10 2024-04-25T01:09:25.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb814072200 con 0x7fb814071df0 2024-04-25T01:09:25.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb81b35e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb8140cfb30 con 0x7fb81406b210 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb8198fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8140add10 0x7fb81415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb8198fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8140add10 0x7fb81415b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60402/0 (socket says 172.21.15.33:60402) 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.372+0000 7fb8198fb640 1 -- 172.21.15.33:0/3146193531 learned_addr learned my addr 172.21.15.33:0/3146193531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8190fa640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb81406b210 0x7fb814156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8188f9640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb814071df0 0x7fb8141580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8198fb640 1 -- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb81406b210 msgr2=0x7fb814156420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8198fb640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb81406b210 0x7fb814156420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8198fb640 1 -- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb814071df0 msgr2=0x7fb8141580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8198fb640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb814071df0 0x7fb8141580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8198fb640 1 -- 172.21.15.33:0/3146193531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb814156960 con 0x7fb8140add10 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8190fa640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb81406b210 0x7fb814156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:25.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8188f9640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb814071df0 0x7fb8141580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:25.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8198fb640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8140add10 0x7fb81415b310 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00cb40 tx=0x7fb80c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb809ffb640 1 -- 172.21.15.33:0/3146193531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c017070 con 0x7fb8140add10 2024-04-25T01:09:25.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb809ffb640 1 -- 172.21.15.33:0/3146193531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb80c00ac40 con 0x7fb8140add10 2024-04-25T01:09:25.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb809ffb640 1 -- 172.21.15.33:0/3146193531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80c012400 con 0x7fb8140add10 2024-04-25T01:09:25.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8141525e0 con 0x7fb8140add10 2024-04-25T01:09:25.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb814153220 con 0x7fb8140add10 2024-04-25T01:09:25.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb814069040 con 0x7fb8140add10 2024-04-25T01:09:25.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb809ffb640 1 -- 172.21.15.33:0/3146193531 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb80c01a070 con 0x7fb8140add10 2024-04-25T01:09:25.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb809ffb640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7f80678f0 0x7fb7f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb809ffb640 1 -- 172.21.15.33:0/3146193531 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb80c085db0 con 0x7fb8140add10 2024-04-25T01:09:25.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8190fa640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7f80678f0 0x7fb7f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.376+0000 7fb8190fa640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7f80678f0 0x7fb7f8069db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb8040af000 tx=0x7fb804008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.380+0000 7fb809ffb640 1 -- 172.21.15.33:0/3146193531 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb8141525e0 con 0x7fb8140add10 2024-04-25T01:09:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.544+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8140cf190 con 0x7fb8140add10 2024-04-25T01:09:25.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.544+0000 7fb809ffb640 1 -- 172.21.15.33:0/3146193531 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fb8140ae170 con 0x7fb8140add10 2024-04-25T01:09:25.538 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:25.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7f80678f0 msgr2=0x7fb7f8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7f80678f0 0x7fb7f8069db0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb8040af000 tx=0x7fb804008040 comp rx=0 tx=0).stop 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8140add10 msgr2=0x7fb81415b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8140add10 0x7fb81415b310 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00cb40 tx=0x7fb80c00a060 comp rx=0 tx=0).stop 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 shutdown_connections 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb7f80678f0 0x7fb7f8069db0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8140add10 0x7fb81415b310 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb814071df0 0x7fb8141580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 --2- 172.21.15.33:0/3146193531 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb81406b210 0x7fb814156420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 >> 172.21.15.33:0/3146193531 conn(0x7fb8140b28c0 msgr2=0x7fb81415c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 shutdown_connections 2024-04-25T01:09:25.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.548+0000 7fb81b35e640 1 -- 172.21.15.33:0/3146193531 wait complete. 2024-04-25T01:09:25.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e4000640 1 Processor -- start 2024-04-25T01:09:25.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e4000640 1 -- start start 2024-04-25T01:09:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e4000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 0x7fc3dc06e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e4000640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3dc06e580 con 0x7fc3dc06c970 2024-04-25T01:09:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e4000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3dc0b3610 con 0x7fc3dc06f0c0 2024-04-25T01:09:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e4000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3dc15a910 con 0x7fc3dc15a530 2024-04-25T01:09:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e159b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc3dc06c970 0x7fc3dc06e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49124/0 (socket says 172.21.15.33:49124) 2024-04-25T01:09:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e159b640 1 -- 172.21.15.33:0/4054861054 learned_addr learned my addr 172.21.15.33:0/4054861054 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 843329196 0 0) 0x7fc3dc06e580 con 0x7fc3dc06c970 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3c4003530 con 0x7fc3dc06c970 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 0x7fc3dc06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 764802193 0 0) 0x7fc3c4003530 con 0x7fc3dc06c970 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 msgr2=0x7fc3dc06e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 0x7fc3dc06e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 >> v1:172.21.15.138:6789/0 conn(0x7fc3dc06f0c0 legacy=0x7fc3dc0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc0b38f0 con 0x7fc3dc06c970 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fc3d0001a00 con 0x7fc3dc06c970 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 >> v1:172.21.15.33:6789/0 conn(0x7fc3dc06c970 legacy=0x7fc3dc06e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c40039f0 0x7fc3c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 0x7fc3c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4004d50 0x7fc3c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c400ba80 con 0x7fc3c40039f0 2024-04-25T01:09:25.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c400bc00 con 0x7fc3c4004d50 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c400be00 con 0x7fc3dc15a530 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c40039f0 0x7fc3c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e159b640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 0x7fc3c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e259d640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4004d50 0x7fc3c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e1d9c640 1 -- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 msgr2=0x7fc3c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3dc15a530 0x7fc3c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e1d9c640 1 -- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4004d50 msgr2=0x7fc3c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3c4004d50 0x7fc3c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.680+0000 7fc3e1d9c640 1 -- 172.21.15.33:0/4054861054 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c4003530 con 0x7fc3c40039f0 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c40039f0 0x7fc3c4003e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009670 tx=0x7fc3d401ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc4bc16671f9ad8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40b4070 con 0x7fc3c40039f0 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d40a75c0 con 0x7fc3c40039f0 2024-04-25T01:09:25.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e0d9a640 1 -- 172.21.15.33:0/4054861054 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40096b0 con 0x7fc3c40039f0 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 -- 172.21.15.33:0/4054861054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c40039f0 msgr2=0x7fc3c4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 --2- 172.21.15.33:0/4054861054 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3c40039f0 0x7fc3c4003e10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009670 tx=0x7fc3d401ec60 comp rx=0 tx=0).stop 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e259d640 1 -- 172.21.15.33:0/4054861054 reap_dead start 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 -- 172.21.15.33:0/4054861054 shutdown_connections 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 -- 172.21.15.33:0/4054861054 >> 172.21.15.33:0/4054861054 conn(0x7fc3dc0715d0 msgr2=0x7fc3dc0719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 -- 172.21.15.33:0/4054861054 shutdown_connections 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 -- 172.21.15.33:0/4054861054 wait complete. 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 Processor -- start 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.684+0000 7fc3e4000640 1 -- start start 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3c40039f0 0x7fc3dc15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc14dc20 0x7fc3dc14d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3dc0ae910 0x7fc3dc0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc0cec00 con 0x7fc3dc0ae910 2024-04-25T01:09:25.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc0cea80 con 0x7fc3dc14dc20 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc3dc0ced80 con 0x7fc3c40039f0 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3dc0ae910 0x7fc3dc0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3dc0ae910 0x7fc3dc0aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60408/0 (socket says 172.21.15.33:60408) 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 -- 172.21.15.33:0/1878027657 learned_addr learned my addr 172.21.15.33:0/1878027657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e159b640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc14dc20 0x7fc3dc14d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3c40039f0 0x7fc3dc15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 -- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3c40039f0 msgr2=0x7fc3dc15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3c40039f0 0x7fc3dc15a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 -- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc14dc20 msgr2=0x7fc3dc14d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc14dc20 0x7fc3dc14d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 -- 172.21.15.33:0/1878027657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc1f6b90 con 0x7fc3dc0ae910 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e159b640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc14dc20 0x7fc3dc14d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:25.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e259d640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3dc0ae910 0x7fc3dc0aed90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc000c10 tx=0x7fc3cc00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3da7fc640 1 -- 172.21.15.33:0/1878027657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc019070 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3da7fc640 1 -- 172.21.15.33:0/1878027657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3cc0052e0 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc15b320 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3da7fc640 1 -- 172.21.15.33:0/1878027657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc003820 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc15b8e0 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc069040 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3da7fc640 1 -- 172.21.15.33:0/1878027657 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3cc01c070 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3da7fc640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3c80678c0 0x7fc3c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3da7fc640 1 -- 172.21.15.33:0/1878027657 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc3cc085460 con 0x7fc3dc0ae910 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.688+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3c80678c0 0x7fc3c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.692+0000 7fc3e1d9c640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3c80678c0 0x7fc3c8069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40b1000 tx=0x7fc3d4010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.692+0000 7fc3da7fc640 1 -- 172.21.15.33:0/1878027657 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc3cc055ba0 con 0x7fc3dc0ae910 2024-04-25T01:09:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.852+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc3dc0748b0 con 0x7fc3dc0ae910 2024-04-25T01:09:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3da7fc640 1 -- 172.21.15.33:0/1878027657 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fc3cc0554a0 con 0x7fc3dc0ae910 2024-04-25T01:09:25.848 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3c80678c0 msgr2=0x7fc3c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3c80678c0 0x7fc3c8069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc3d40b1000 tx=0x7fc3d4010040 comp rx=0 tx=0).stop 2024-04-25T01:09:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3dc0ae910 msgr2=0x7fc3dc0aed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3dc0ae910 0x7fc3dc0aed90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc000c10 tx=0x7fc3cc00e8a0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 shutdown_connections 2024-04-25T01:09:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc3c80678c0 0x7fc3c8069d80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3dc0ae910 0x7fc3dc0aed90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3dc14dc20 0x7fc3dc14d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 --2- 172.21.15.33:0/1878027657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc3c40039f0 0x7fc3dc15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 >> 172.21.15.33:0/1878027657 conn(0x7fc3dc0715d0 msgr2=0x7fc3dc157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 shutdown_connections 2024-04-25T01:09:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.856+0000 7fc3e4000640 1 -- 172.21.15.33:0/1878027657 wait complete. 2024-04-25T01:09:25.867 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.996+0000 7f7d740d9640 1 Processor -- start 2024-04-25T01:09:25.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:25.996+0000 7f7d740d9640 1 -- start start 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d740d9640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f7d6c0d1f60 0x7f7d6c0d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d740d9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d740d9640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d6c0b6cc0 con 0x7f7d6c15e0d0 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d740d9640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f7d6c0cfb00 con 0x7f7d6c0d1f60 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d740d9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c0b7580 con 0x7f7d6c0b9a20 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71e75640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f7d6c15e0d0 0x7f7d6c0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49126/0 (socket says 172.21.15.33:49126) 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71e75640 1 -- 172.21.15.33:0/3156305858 learned_addr learned my addr 172.21.15.33:0/3156305858 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71674640 1 --2- 172.21.15.33:0/3156305858 >> v2:172.21.15.33:3300/0 conn(0x7f7d6c0d1f60 0x7f7d6c0d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d72676640 1 --2- 172.21.15.33:0/3156305858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d70e73640 1 -- 172.21.15.33:0/3156305858 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1079838874 0 0) 0x7f7d6c0b6cc0 con 0x7f7d6c15e0d0 2024-04-25T01:09:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d70e73640 1 -- 172.21.15.33:0/3156305858 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d58003530 con 0x7f7d6c15e0d0 2024-04-25T01:09:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71674640 1 -- 172.21.15.33:0/3156305858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 msgr2=0x7f7d6c0d28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71674640 1 --2- 172.21.15.33:0/3156305858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0d28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71674640 1 -- 172.21.15.33:0/3156305858 >> v1:172.21.15.33:6789/0 conn(0x7f7d6c15e0d0 legacy=0x7f7d6c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71674640 1 -- 172.21.15.33:0/3156305858 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0b6f30 con 0x7f7d6c0d1f60 2024-04-25T01:09:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d71674640 1 --2- 172.21.15.33:0/3156305858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c0d23b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0097c0 tx=0x7f7d5c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9b5c63e533a5be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d70e73640 1 -- 172.21.15.33:0/3156305858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0aa5f0 con 0x7f7d6c0d1f60 2024-04-25T01:09:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d70e73640 1 -- 172.21.15.33:0/3156305858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c0af070 con 0x7f7d6c0d1f60 2024-04-25T01:09:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.000+0000 7f7d70e73640 1 -- 172.21.15.33:0/3156305858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0aa990 con 0x7f7d6c0d1f60 2024-04-25T01:09:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- 172.21.15.33:0/3156305858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 msgr2=0x7f7d6c0d23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 --2- 172.21.15.33:0/3156305858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c0d23b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0097c0 tx=0x7f7d5c01c470 comp rx=0 tx=0).stop 2024-04-25T01:09:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- 172.21.15.33:0/3156305858 shutdown_connections 2024-04-25T01:09:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 --2- 172.21.15.33:0/3156305858 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0d28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 --2- 172.21.15.33:0/3156305858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c0d23b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- 172.21.15.33:0/3156305858 >> 172.21.15.33:0/3156305858 conn(0x7f7d6c0d3dd0 msgr2=0x7f7d6c0d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- 172.21.15.33:0/3156305858 shutdown_connections 2024-04-25T01:09:25.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- 172.21.15.33:0/3156305858 wait complete. 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 Processor -- start 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- start start 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d6c15e0d0 0x7f7d6c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c16a690 con 0x7f7d6c0d1f60 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6c16a510 con 0x7f7d6c15e0d0 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6c16a810 con 0x7f7d6c0b9a20 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:25.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c15a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60426/0 (socket says 172.21.15.33:60426) 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 -- 172.21.15.33:0/529416698 learned_addr learned my addr 172.21.15.33:0/529416698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71e75640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d72676640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d6c15e0d0 0x7f7d6c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 -- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 msgr2=0x7f7d6c0cb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0cb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 -- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d6c15e0d0 msgr2=0x7f7d6c15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d6c15e0d0 0x7f7d6c15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 -- 172.21.15.33:0/529416698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6c0a0ce0 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d71674640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c15a4e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0ac040 tx=0x7f7d5c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d627fc640 1 -- 172.21.15.33:0/529416698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c003a30 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d627fc640 1 -- 172.21.15.33:0/529416698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5c003bd0 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6c09dd30 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d627fc640 1 -- 172.21.15.33:0/529416698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0b33f0 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.004+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6c09e2f0 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.008+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d6c069040 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.008+0000 7f7d627fc640 1 -- 172.21.15.33:0/529416698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7d5c0c2060 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.008+0000 7f7d627fc640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7d500678f0 0x7f7d50069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.008+0000 7f7d627fc640 1 -- 172.21.15.33:0/529416698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7d5c11fb10 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.008+0000 7f7d71e75640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7d500678f0 0x7f7d50069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.008+0000 7f7d71e75640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7d500678f0 0x7f7d50069db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005890 tx=0x7f7d68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.012+0000 7f7d627fc640 1 -- 172.21.15.33:0/529416698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7d5c016090 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.172+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7d6c0d43c0 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.172+0000 7f7d627fc640 1 -- 172.21.15.33:0/529416698 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f7d5c0f0250 con 0x7f7d6c0d1f60 2024-04-25T01:09:26.166 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7d500678f0 msgr2=0x7f7d50069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7d500678f0 0x7f7d50069db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7d68005890 tx=0x7f7d68009040 comp rx=0 tx=0).stop 2024-04-25T01:09:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 msgr2=0x7f7d6c15a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c15a4e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0ac040 tx=0x7f7d5c019210 comp rx=0 tx=0).stop 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 shutdown_connections 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f7d500678f0 0x7f7d50069db0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d6c15e0d0 0x7f7d6c15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d6c0d1f60 0x7f7d6c15a4e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 --2- 172.21.15.33:0/529416698 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7d6c0b9a20 0x7f7d6c0cb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 >> 172.21.15.33:0/529416698 conn(0x7f7d6c0d3dd0 msgr2=0x7f7d6c0b7a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 shutdown_connections 2024-04-25T01:09:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.176+0000 7f7d740d9640 1 -- 172.21.15.33:0/529416698 wait complete. 2024-04-25T01:09:26.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f879a909640 1 Processor -- start 2024-04-25T01:09:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f879a909640 1 -- start start 2024-04-25T01:09:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f879a909640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f879415c040 0x7f8794072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f879a909640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87940af620 0x7f8794072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f879a909640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87940b2a20 con 0x7f8794073390 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f879a909640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f87940b0150 con 0x7f87940af620 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f879a909640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f87940b0430 con 0x7f879415c040 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f87937fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87940af620 0x7f8794072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f87937fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f87940af620 0x7f8794072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60432/0 (socket says 172.21.15.33:60432) 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f87937fe640 1 -- 172.21.15.33:0/301697122 learned_addr learned my addr 172.21.15.33:0/301697122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f8793fff640 1 --2- 172.21.15.33:0/301697122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f879415c040 0x7f8794072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f87937fe640 1 -- 172.21.15.33:0/301697122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f879415c040 msgr2=0x7f8794072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f87937fe640 1 --2- 172.21.15.33:0/301697122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f879415c040 0x7f8794072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f87937fe640 1 -- 172.21.15.33:0/301697122 >> v1:172.21.15.138:6789/0 conn(0x7f8794073390 legacy=0x7f87940b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f87937fe640 1 -- 172.21.15.33:0/301697122 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87940b0710 con 0x7f87940af620 2024-04-25T01:09:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.312+0000 7f8793fff640 1 --2- 172.21.15.33:0/301697122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f879415c040 0x7f8794072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f87937fe640 1 --2- 172.21.15.33:0/301697122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87940af620 0x7f8794072e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8788009870 tx=0x7f878801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c9fe17ff69d0432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f8792ffd640 1 -- 172.21.15.33:0/301697122 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87880b0070 con 0x7f87940af620 2024-04-25T01:09:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f8792ffd640 1 -- 172.21.15.33:0/301697122 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87880ab600 con 0x7f87940af620 2024-04-25T01:09:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f8792ffd640 1 -- 172.21.15.33:0/301697122 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87880ab9d0 con 0x7f87940af620 2024-04-25T01:09:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 -- 172.21.15.33:0/301697122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87940af620 msgr2=0x7f8794072e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 --2- 172.21.15.33:0/301697122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87940af620 0x7f8794072e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8788009870 tx=0x7f878801c560 comp rx=0 tx=0).stop 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 -- 172.21.15.33:0/301697122 shutdown_connections 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 --2- 172.21.15.33:0/301697122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87940af620 0x7f8794072e50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 --2- 172.21.15.33:0/301697122 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f879415c040 0x7f8794072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 -- 172.21.15.33:0/301697122 >> 172.21.15.33:0/301697122 conn(0x7f87940b1e60 msgr2=0x7f87940b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 -- 172.21.15.33:0/301697122 shutdown_connections 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 -- 172.21.15.33:0/301697122 wait complete. 2024-04-25T01:09:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 Processor -- start 2024-04-25T01:09:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 -- start start 2024-04-25T01:09:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.316+0000 7f879a909640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8794073390 0x7f87941532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87940af620 0x7f8794165580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879415c040 0x7f87940cf230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8794168a60 con 0x7f879415c040 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8794168be0 con 0x7f8794073390 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8794168d60 con 0x7f87940af620 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879415c040 0x7f87940cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879415c040 0x7f87940cf230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60446/0 (socket says 172.21.15.33:60446) 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 -- 172.21.15.33:0/2783348487 learned_addr learned my addr 172.21.15.33:0/2783348487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f87937fe640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87940af620 0x7f8794165580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8793fff640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8794073390 0x7f87941532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 -- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87940af620 msgr2=0x7f8794165580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87940af620 0x7f8794165580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 -- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8794073390 msgr2=0x7f87941532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8794073390 0x7f87941532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 -- 172.21.15.33:0/2783348487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8794072180 con 0x7f879415c040 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8793fff640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8794073390 0x7f87941532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f87937fe640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87940af620 0x7f8794165580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8798ea6640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879415c040 0x7f87940cf230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c730 tx=0x7f878c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8790ff9640 1 -- 172.21.15.33:0/2783348487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c003b80 con 0x7f879415c040 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8790ff9640 1 -- 172.21.15.33:0/2783348487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c003d20 con 0x7f879415c040 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87940731c0 con 0x7f879415c040 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8790ff9640 1 -- 172.21.15.33:0/2783348487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c00aa30 con 0x7f879415c040 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87940cfd60 con 0x7f879415c040 2024-04-25T01:09:26.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8794069040 con 0x7f879415c040 2024-04-25T01:09:26.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8790ff9640 1 -- 172.21.15.33:0/2783348487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f878c00abd0 con 0x7f879415c040 2024-04-25T01:09:26.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8790ff9640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87800678c0 0x7f8780069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8790ff9640 1 -- 172.21.15.33:0/2783348487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f878c085fe0 con 0x7f879415c040 2024-04-25T01:09:26.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8793fff640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87800678c0 0x7f8780069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.320+0000 7f8793fff640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87800678c0 0x7f8780069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8784009980 tx=0x7f8784008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.324+0000 7f8790ff9640 1 -- 172.21.15.33:0/2783348487 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f878c056680 con 0x7f879415c040 2024-04-25T01:09:26.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.480+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f87940cf010 con 0x7f879415c040 2024-04-25T01:09:26.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f8790ff9640 1 -- 172.21.15.33:0/2783348487 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f878c055f80 con 0x7f879415c040 2024-04-25T01:09:26.475 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:26.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87800678c0 msgr2=0x7f8780069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87800678c0 0x7f8780069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8784009980 tx=0x7f8784008040 comp rx=0 tx=0).stop 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879415c040 msgr2=0x7f87940cf230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879415c040 0x7f87940cf230 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c730 tx=0x7f878c00cbf0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 shutdown_connections 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f87800678c0 0x7f8780069d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f879415c040 0x7f87940cf230 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f87940af620 0x7f8794165580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 --2- 172.21.15.33:0/2783348487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8794073390 0x7f87941532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 >> 172.21.15.33:0/2783348487 conn(0x7f87940b1e60 msgr2=0x7f879415ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 shutdown_connections 2024-04-25T01:09:26.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.484+0000 7f879a909640 1 -- 172.21.15.33:0/2783348487 wait complete. 2024-04-25T01:09:26.493 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64bbe3640 1 Processor -- start 2024-04-25T01:09:26.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64bbe3640 1 -- start start 2024-04-25T01:09:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64bbe3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa644145470 0x7fa64415c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64bbe3640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa6440b5530 0x7fa6440cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64bbe3640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64415f9a0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64bbe3640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa64415f750 con 0x7fa6440b5530 2024-04-25T01:09:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64bbe3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa64415fc80 con 0x7fa644145470 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64917e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa6440cf0e0 0x7fa6440b4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49138/0 (socket says 172.21.15.33:49138) 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64917e640 1 -- 172.21.15.33:0/335277755 learned_addr learned my addr 172.21.15.33:0/335277755 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64a180640 1 --2- 172.21.15.33:0/335277755 >> v2:172.21.15.33:3300/0 conn(0x7fa6440b5530 0x7fa6440cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64997f640 1 --2- 172.21.15.33:0/335277755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa644145470 0x7fa64415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64897d640 1 -- 172.21.15.33:0/335277755 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1690489085 0 0) 0x7fa64415f9a0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64897d640 1 -- 172.21.15.33:0/335277755 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa630003530 con 0x7fa6440cf0e0 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64a180640 1 -- 172.21.15.33:0/335277755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa644145470 msgr2=0x7fa64415c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64a180640 1 --2- 172.21.15.33:0/335277755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa644145470 0x7fa64415c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64a180640 1 -- 172.21.15.33:0/335277755 >> v1:172.21.15.33:6789/0 conn(0x7fa6440cf0e0 legacy=0x7fa6440b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.624+0000 7fa64a180640 1 -- 172.21.15.33:0/335277755 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64415ff60 con 0x7fa6440b5530 2024-04-25T01:09:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64a180640 1 --2- 172.21.15.33:0/335277755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440b5530 0x7fa6440cce00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009d90 tx=0x7fa63c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9027d4460e0259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64897d640 1 -- 172.21.15.33:0/335277755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0b2070 con 0x7fa6440b5530 2024-04-25T01:09:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64897d640 1 -- 172.21.15.33:0/335277755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63c0a5940 con 0x7fa6440b5530 2024-04-25T01:09:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64897d640 1 -- 172.21.15.33:0/335277755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c0a5cf0 con 0x7fa6440b5530 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 -- 172.21.15.33:0/335277755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440b5530 msgr2=0x7fa6440cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/335277755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440b5530 0x7fa6440cce00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa63c009d90 tx=0x7fa63c01c970 comp rx=0 tx=0).stop 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 -- 172.21.15.33:0/335277755 shutdown_connections 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/335277755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440b5530 0x7fa6440cce00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/335277755 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa644145470 0x7fa64415c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 -- 172.21.15.33:0/335277755 >> 172.21.15.33:0/335277755 conn(0x7fa6440b1810 msgr2=0x7fa6440b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 -- 172.21.15.33:0/335277755 shutdown_connections 2024-04-25T01:09:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 -- 172.21.15.33:0/335277755 wait complete. 2024-04-25T01:09:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 Processor -- start 2024-04-25T01:09:26.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.628+0000 7fa64bbe3640 1 -- start start 2024-04-25T01:09:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6440b5530 0x7fa644156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440cf0e0 0x7fa644158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa644145470 0x7fa644142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa644168c70 con 0x7fa6440cf0e0 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa644168af0 con 0x7fa644145470 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa644168df0 con 0x7fa6440b5530 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440cf0e0 0x7fa644158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440cf0e0 0x7fa644158fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60458/0 (socket says 172.21.15.33:60458) 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 -- 172.21.15.33:0/625249569 learned_addr learned my addr 172.21.15.33:0/625249569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64997f640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6440b5530 0x7fa644156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64a180640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa644145470 0x7fa644142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 -- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6440b5530 msgr2=0x7fa644156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6440b5530 0x7fa644156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 -- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa644145470 msgr2=0x7fa644142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa644145470 0x7fa644142840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 -- 172.21.15.33:0/625249569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa644157f20 con 0x7fa6440cf0e0 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64997f640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6440b5530 0x7fa644156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64917e640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440cf0e0 0x7fa644158fc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c3c0 tx=0x7fa63400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa63a7fc640 1 -- 172.21.15.33:0/625249569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa634017070 con 0x7fa6440cf0e0 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa63a7fc640 1 -- 172.21.15.33:0/625249569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa634005cf0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa63a7fc640 1 -- 172.21.15.33:0/625249569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6340072f0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa644158200 con 0x7fa6440cf0e0 2024-04-25T01:09:26.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441587c0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa644069040 con 0x7fa6440cf0e0 2024-04-25T01:09:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa63a7fc640 1 -- 172.21.15.33:0/625249569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa63401a070 con 0x7fa6440cf0e0 2024-04-25T01:09:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa63a7fc640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6280678c0 0x7fa628069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa63a7fc640 1 -- 172.21.15.33:0/625249569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa6340854c0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.632+0000 7fa64997f640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6280678c0 0x7fa628069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:26.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.636+0000 7fa64997f640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6280678c0 0x7fa628069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa644144e90 tx=0x7fa640008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:26.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.636+0000 7fa63a7fc640 1 -- 172.21.15.33:0/625249569 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa634055be0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.804+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa644071400 con 0x7fa6440cf0e0 2024-04-25T01:09:26.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.804+0000 7fa63a7fc640 1 -- 172.21.15.33:0/625249569 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fa6340554e0 con 0x7fa6440cf0e0 2024-04-25T01:09:26.797 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:26.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.804+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6280678c0 msgr2=0x7fa628069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.804+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6280678c0 0x7fa628069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa644144e90 tx=0x7fa640008040 comp rx=0 tx=0).stop 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.804+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440cf0e0 msgr2=0x7fa644158fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.804+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440cf0e0 0x7fa644158fc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c3c0 tx=0x7fa63400c880 comp rx=0 tx=0).stop 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 shutdown_connections 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa6280678c0 0x7fa628069d80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa644145470 0x7fa644142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6440cf0e0 0x7fa644158fc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 --2- 172.21.15.33:0/625249569 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa6440b5530 0x7fa644156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 >> 172.21.15.33:0/625249569 conn(0x7fa6440b1810 msgr2=0x7fa6440b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 shutdown_connections 2024-04-25T01:09:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:26.808+0000 7fa64bbe3640 1 -- 172.21.15.33:0/625249569 wait complete. 2024-04-25T01:09:26.815 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-04-25T01:09:26.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:26.815 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T01:09:26.828 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:26.829 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:09:26.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:26.829 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T01:09:26.876 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:09:26.876 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:26.882 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:09:26.883 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:26.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 Processor -- start 2024-04-25T01:09:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 -- start start 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c0b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f728c15f220 0x7f728c06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f728c06e050 0x7f728c06fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f728c0b6060 con 0x7f728c15f220 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f728c0b6340 con 0x7f728c06e050 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7294154640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f728c0b6620 con 0x7f728c06dc70 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f728c15f220 0x7f728c06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f728c15f220 0x7f728c06f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60462/0 (socket says 172.21.15.33:60462) 2024-04-25T01:09:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 -- 172.21.15.33:0/2327475679 learned_addr learned my addr 172.21.15.33:0/2327475679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f7291ef0640 1 --2- 172.21.15.33:0/2327475679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c0b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72926f1640 1 --2- 172.21.15.33:0/2327475679 >> v2:172.21.15.138:3300/0 conn(0x7f728c06e050 0x7f728c06fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 -- 172.21.15.33:0/2327475679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 msgr2=0x7f728c0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 --2- 172.21.15.33:0/2327475679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c0b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 -- 172.21.15.33:0/2327475679 >> v2:172.21.15.138:3300/0 conn(0x7f728c06e050 msgr2=0x7f728c06fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 --2- 172.21.15.33:0/2327475679 >> v2:172.21.15.138:3300/0 conn(0x7f728c06e050 0x7f728c06fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.060+0000 7f72916ef640 1 -- 172.21.15.33:0/2327475679 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c0b6900 con 0x7f728c15f220 2024-04-25T01:09:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f72916ef640 1 --2- 172.21.15.33:0/2327475679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c15f220 0x7f728c06f8a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f728c0b4e80 tx=0x7f727c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6784ee48f25dee88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7290eee640 1 -- 172.21.15.33:0/2327475679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0ab5f0 con 0x7f728c15f220 2024-04-25T01:09:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7290eee640 1 -- 172.21.15.33:0/2327475679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727c0b0070 con 0x7f728c15f220 2024-04-25T01:09:27.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7290eee640 1 -- 172.21.15.33:0/2327475679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0ab9b0 con 0x7f728c15f220 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- 172.21.15.33:0/2327475679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c15f220 msgr2=0x7f728c06f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 --2- 172.21.15.33:0/2327475679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c15f220 0x7f728c06f8a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f728c0b4e80 tx=0x7f727c01c520 comp rx=0 tx=0).stop 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- 172.21.15.33:0/2327475679 shutdown_connections 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 --2- 172.21.15.33:0/2327475679 >> v2:172.21.15.138:3300/0 conn(0x7f728c06e050 0x7f728c06fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 --2- 172.21.15.33:0/2327475679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c15f220 0x7f728c06f8a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 --2- 172.21.15.33:0/2327475679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c0b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- 172.21.15.33:0/2327475679 >> 172.21.15.33:0/2327475679 conn(0x7f728c0734c0 msgr2=0x7f728c09a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- 172.21.15.33:0/2327475679 shutdown_connections 2024-04-25T01:09:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- 172.21.15.33:0/2327475679 wait complete. 2024-04-25T01:09:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 Processor -- start 2024-04-25T01:09:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- start start 2024-04-25T01:09:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c1545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c06e050 0x7f728c1485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c15f220 0x7f728c0aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f728c162650 con 0x7f728c06e050 2024-04-25T01:09:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f728c1624d0 con 0x7f728c15f220 2024-04-25T01:09:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7294154640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f728c1627d0 con 0x7f728c06dc70 2024-04-25T01:09:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f72916ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c06e050 0x7f728c1485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f72916ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c06e050 0x7f728c1485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60470/0 (socket says 172.21.15.33:60470) 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f72916ef640 1 -- 172.21.15.33:0/2190949124 learned_addr learned my addr 172.21.15.33:0/2190949124 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f72926f1640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c15f220 0x7f728c0aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.064+0000 7f7291ef0640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c1545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72916ef640 1 -- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 msgr2=0x7f728c1545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72916ef640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c1545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72916ef640 1 -- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c15f220 msgr2=0x7f728c0aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72916ef640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c15f220 0x7f728c0aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72916ef640 1 -- 172.21.15.33:0/2190949124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f728c15bd80 con 0x7f728c06e050 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72926f1640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c15f220 0x7f728c0aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f7291ef0640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72916ef640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c06e050 0x7f728c1485c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ad040 tx=0x7f727c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72827fc640 1 -- 172.21.15.33:0/2190949124 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0bf610 con 0x7f728c06e050 2024-04-25T01:09:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72827fc640 1 -- 172.21.15.33:0/2190949124 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f727c0bf7b0 con 0x7f728c06e050 2024-04-25T01:09:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f728c148f50 con 0x7f728c06e050 2024-04-25T01:09:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72827fc640 1 -- 172.21.15.33:0/2190949124 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f727c0bfba0 con 0x7f728c06e050 2024-04-25T01:09:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f728c149510 con 0x7f728c06e050 2024-04-25T01:09:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f728c069040 con 0x7f728c06e050 2024-04-25T01:09:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72827fc640 1 -- 172.21.15.33:0/2190949124 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f727c0bd020 con 0x7f728c06e050 2024-04-25T01:09:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72827fc640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f72700678c0 0x7f7270069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f72827fc640 1 -- 172.21.15.33:0/2190949124 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f727c1207b0 con 0x7f728c06e050 2024-04-25T01:09:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f7291ef0640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f72700678c0 0x7f7270069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.068+0000 7f7291ef0640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f72700678c0 0x7f7270069d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7288009a90 tx=0x7f7288008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.072+0000 7f72827fc640 1 -- 172.21.15.33:0/2190949124 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f727c016110 con 0x7f728c06e050 2024-04-25T01:09:27.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.232+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f728c0d2a90 con 0x7f728c06e050 2024-04-25T01:09:27.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.232+0000 7f72827fc640 1 -- 172.21.15.33:0/2190949124 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7f727c0f1000 con 0x7f728c06e050 2024-04-25T01:09:27.226 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:27.227 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f72700678c0 msgr2=0x7f7270069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f72700678c0 0x7f7270069d80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7288009a90 tx=0x7f7288008040 comp rx=0 tx=0).stop 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c06e050 msgr2=0x7f728c1485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c06e050 0x7f728c1485c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ad040 tx=0x7f727c0192c0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 shutdown_connections 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f72700678c0 0x7f7270069d80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c15f220 0x7f728c0aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c06e050 0x7f728c1485c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 --2- 172.21.15.33:0/2190949124 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f728c06dc70 0x7f728c1545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 >> 172.21.15.33:0/2190949124 conn(0x7f728c0734c0 msgr2=0x7f728c158fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:27.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 shutdown_connections 2024-04-25T01:09:27.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.236+0000 7f7294154640 1 -- 172.21.15.33:0/2190949124 wait complete. 2024-04-25T01:09:27.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:27.245 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6cf5e640 1 Processor -- start 2024-04-25T01:09:27.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6cf5e640 1 -- start start 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6cf5e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fee6806ecc0 0x7fee680b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6cf5e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee68072410 0x7fee680b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6cf5e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee680b4a20 con 0x7fee6815ae20 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6cf5e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fee680b2bb0 con 0x7fee6806ecc0 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6cf5e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee680b2e90 con 0x7fee68072410 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee5dd9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fee6806ecc0 0x7fee680b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6659c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fee6815ae20 0x7fee6806e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49146/0 (socket says 172.21.15.33:49146) 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee6659c640 1 -- 172.21.15.33:0/2719177354 learned_addr learned my addr 172.21.15.33:0/2719177354 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee66d9d640 1 --2- 172.21.15.33:0/2719177354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee68072410 0x7fee680b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee5dd9b640 1 -- 172.21.15.33:0/2719177354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee68072410 msgr2=0x7fee680b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee5dd9b640 1 --2- 172.21.15.33:0/2719177354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee68072410 0x7fee680b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee5dd9b640 1 -- 172.21.15.33:0/2719177354 >> v1:172.21.15.33:6789/0 conn(0x7fee6815ae20 legacy=0x7fee6806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee5dd9b640 1 -- 172.21.15.33:0/2719177354 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee680b3170 con 0x7fee6806ecc0 2024-04-25T01:09:27.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee65d9b640 1 -- 172.21.15.33:0/2719177354 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4004809730 0 0) 0x7fee680b4a20 con 0x7fee6815ae20 2024-04-25T01:09:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee5dd9b640 1 --2- 172.21.15.33:0/2719177354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee680b3690 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fee500097c0 tx=0x7fee5001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ff3c8e6bc5bd24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee65d9b640 1 -- 172.21.15.33:0/2719177354 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500aa620 con 0x7fee6806ecc0 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee65d9b640 1 -- 172.21.15.33:0/2719177354 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee500af070 con 0x7fee6806ecc0 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.368+0000 7fee65d9b640 1 -- 172.21.15.33:0/2719177354 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee500aa9e0 con 0x7fee6806ecc0 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/2719177354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 msgr2=0x7fee680b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/2719177354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee680b3690 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fee500097c0 tx=0x7fee5001c4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/2719177354 shutdown_connections 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/2719177354 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee68072410 0x7fee680b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/2719177354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee680b3690 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/2719177354 >> 172.21.15.33:0/2719177354 conn(0x7fee680b1810 msgr2=0x7fee680b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/2719177354 shutdown_connections 2024-04-25T01:09:27.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/2719177354 wait complete. 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 Processor -- start 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- start start 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee68141590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68072410 0x7fee680aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee6815ae20 0x7fee68154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee680cdad0 con 0x7fee6806ecc0 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fee680cd950 con 0x7fee68072410 2024-04-25T01:09:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fee680cdc50 con 0x7fee6815ae20 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee68141590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee68141590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60486/0 (socket says 172.21.15.33:60486) 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 -- 172.21.15.33:0/3006555085 learned_addr learned my addr 172.21.15.33:0/3006555085 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 -- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee6815ae20 msgr2=0x7fee68154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee6815ae20 0x7fee68154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 -- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68072410 msgr2=0x7fee680aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68072410 0x7fee680aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 -- 172.21.15.33:0/3006555085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee68072b00 con 0x7fee6806ecc0 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6659c640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee68141590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c480 tx=0x7fee5800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee5f7fe640 1 -- 172.21.15.33:0/3006555085 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58018070 con 0x7fee6806ecc0 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee5f7fe640 1 -- 172.21.15.33:0/3006555085 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee58007500 con 0x7fee6806ecc0 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee6815e890 con 0x7fee6806ecc0 2024-04-25T01:09:27.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee5f7fe640 1 -- 172.21.15.33:0/3006555085 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee58004a30 con 0x7fee6806ecc0 2024-04-25T01:09:27.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee6815ee50 con 0x7fee6806ecc0 2024-04-25T01:09:27.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.372+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee68069040 con 0x7fee6806ecc0 2024-04-25T01:09:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.376+0000 7fee5f7fe640 1 -- 172.21.15.33:0/3006555085 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fee5801b070 con 0x7fee6806ecc0 2024-04-25T01:09:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.376+0000 7fee5f7fe640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee480678f0 0x7fee48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.376+0000 7fee5f7fe640 1 -- 172.21.15.33:0/3006555085 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fee580852c0 con 0x7fee6806ecc0 2024-04-25T01:09:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.376+0000 7fee5dd9b640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee480678f0 0x7fee48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.376+0000 7fee5dd9b640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee480678f0 0x7fee48069db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee500ae000 tx=0x7fee50004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.380+0000 7fee5f7fe640 1 -- 172.21.15.33:0/3006555085 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fee58055a90 con 0x7fee6806ecc0 2024-04-25T01:09:27.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.536+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee6806ce50 con 0x7fee6806ecc0 2024-04-25T01:09:27.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.536+0000 7fee5f7fe640 1 -- 172.21.15.33:0/3006555085 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fee58055390 con 0x7fee6806ecc0 2024-04-25T01:09:27.529 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee480678f0 msgr2=0x7fee48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee480678f0 0x7fee48069db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fee500ae000 tx=0x7fee50004740 comp rx=0 tx=0).stop 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 msgr2=0x7fee68141590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee68141590 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c480 tx=0x7fee5800c940 comp rx=0 tx=0).stop 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 shutdown_connections 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fee480678f0 0x7fee48069db0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fee6815ae20 0x7fee68154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee68072410 0x7fee680aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 --2- 172.21.15.33:0/3006555085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee6806ecc0 0x7fee68141590 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 >> 172.21.15.33:0/3006555085 conn(0x7fee680b1810 msgr2=0x7fee680b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 shutdown_connections 2024-04-25T01:09:27.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.540+0000 7fee6cf5e640 1 -- 172.21.15.33:0/3006555085 wait complete. 2024-04-25T01:09:27.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:27.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.672+0000 7f774e9c3640 1 Processor -- start 2024-04-25T01:09:27.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.672+0000 7f774e9c3640 1 -- start start 2024-04-25T01:09:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f774e9c3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77480b5580 0x7f77480b5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f774e9c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f774e9c3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77480b05a0 con 0x7f77480d1b50 2024-04-25T01:09:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f774e9c3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f77480b5ea0 con 0x7f77480b5580 2024-04-25T01:09:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f774e9c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f774815d380 con 0x7f77480b50d0 2024-04-25T01:09:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f773f7fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f774cf60640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f77480d1b50 0x7f77480d2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:53362/0 (socket says 172.21.15.33:53362) 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f7747fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77480b5580 0x7f77480b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f774cf60640 1 -- 172.21.15.33:0/4068279426 learned_addr learned my addr 172.21.15.33:0/4068279426 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f77477fe640 1 -- 172.21.15.33:0/4068279426 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2653145109 0 0) 0x7f77400014a0 con 0x7f77480d1b50 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f77477fe640 1 -- 172.21.15.33:0/4068279426 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7734003530 con 0x7f77480d1b50 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f773f7fe640 1 -- 172.21.15.33:0/4068279426 >> v2:172.21.15.138:3300/0 conn(0x7f77480b5580 msgr2=0x7f77480b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f773f7fe640 1 --2- 172.21.15.33:0/4068279426 >> v2:172.21.15.138:3300/0 conn(0x7f77480b5580 0x7f77480b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f773f7fe640 1 -- 172.21.15.33:0/4068279426 >> v1:172.21.15.138:6789/0 conn(0x7f77480d1b50 legacy=0x7f77480d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.676+0000 7f773f7fe640 1 -- 172.21.15.33:0/4068279426 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77480af500 con 0x7f77480b50d0 2024-04-25T01:09:27.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f773f7fe640 1 --2- 172.21.15.33:0/4068279426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774815ce40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77300097c0 tx=0x7f773001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9f2d04800c8df55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f77477fe640 1 -- 172.21.15.33:0/4068279426 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300b0070 con 0x7f77480b50d0 2024-04-25T01:09:27.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f77477fe640 1 -- 172.21.15.33:0/4068279426 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77300ab600 con 0x7f77480b50d0 2024-04-25T01:09:27.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f77477fe640 1 -- 172.21.15.33:0/4068279426 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300ab9b0 con 0x7f77480b50d0 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 -- 172.21.15.33:0/4068279426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 msgr2=0x7f774815ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 --2- 172.21.15.33:0/4068279426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774815ce40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77300097c0 tx=0x7f773001c4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 -- 172.21.15.33:0/4068279426 shutdown_connections 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 --2- 172.21.15.33:0/4068279426 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774815ce40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 --2- 172.21.15.33:0/4068279426 >> v2:172.21.15.138:3300/0 conn(0x7f77480b5580 0x7f77480b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 -- 172.21.15.33:0/4068279426 >> 172.21.15.33:0/4068279426 conn(0x7f774809a9d0 msgr2=0x7f774809ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 -- 172.21.15.33:0/4068279426 shutdown_connections 2024-04-25T01:09:27.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 -- 172.21.15.33:0/4068279426 wait complete. 2024-04-25T01:09:27.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 Processor -- start 2024-04-25T01:09:27.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 -- start start 2024-04-25T01:09:27.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.680+0000 7f774e9c3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77480b5580 0x7f7748144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77480d1b50 0x7f774815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77480af1b0 con 0x7f77480b5580 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77480b0080 con 0x7f77480d1b50 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f774815e5e0 con 0x7f77480b50d0 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77480b5580 0x7f7748144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77480b5580 0x7f7748144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60490/0 (socket says 172.21.15.33:60490) 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 -- 172.21.15.33:0/3043654903 learned_addr learned my addr 172.21.15.33:0/3043654903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f7747fff640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774cf60640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77480d1b50 0x7f774815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 -- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 msgr2=0x7f774806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774806d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 -- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77480d1b50 msgr2=0x7f774815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77480d1b50 0x7f774815b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 -- 172.21.15.33:0/3043654903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774806c700 con 0x7f77480b5580 2024-04-25T01:09:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f773f7fe640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77480b5580 0x7f7748144230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7730000c10 tx=0x7f7730019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f77457fa640 1 -- 172.21.15.33:0/3043654903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300b0040 con 0x7f77480b5580 2024-04-25T01:09:27.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f77457fa640 1 -- 172.21.15.33:0/3043654903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77300bb070 con 0x7f77480b5580 2024-04-25T01:09:27.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774806c9e0 con 0x7f77480b5580 2024-04-25T01:09:27.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f77457fa640 1 -- 172.21.15.33:0/3043654903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300ab9b0 con 0x7f77480b5580 2024-04-25T01:09:27.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774806cfa0 con 0x7f77480b5580 2024-04-25T01:09:27.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7748069040 con 0x7f77480b5580 2024-04-25T01:09:27.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f77457fa640 1 -- 172.21.15.33:0/3043654903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f77300b3070 con 0x7f77480b5580 2024-04-25T01:09:27.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f77457fa640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77280678c0 0x7f7728069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f77457fa640 1 -- 172.21.15.33:0/3043654903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f773011fe20 con 0x7f77480b5580 2024-04-25T01:09:27.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.684+0000 7f7747fff640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77280678c0 0x7f7728069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.688+0000 7f7747fff640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77280678c0 0x7f7728069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77481584d0 tx=0x7f7738008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.688+0000 7f77457fa640 1 -- 172.21.15.33:0/3043654903 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7730016090 con 0x7f77480b5580 2024-04-25T01:09:27.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.844+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f774806dce0 con 0x7f77480b5580 2024-04-25T01:09:27.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.844+0000 7f77457fa640 1 -- 172.21.15.33:0/3043654903 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f77300f04c0 con 0x7f77480b5580 2024-04-25T01:09:27.837 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77280678c0 msgr2=0x7f7728069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77280678c0 0x7f7728069d80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f77481584d0 tx=0x7f7738008040 comp rx=0 tx=0).stop 2024-04-25T01:09:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77480b5580 msgr2=0x7f7748144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77480b5580 0x7f7748144230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7730000c10 tx=0x7f7730019240 comp rx=0 tx=0).stop 2024-04-25T01:09:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 shutdown_connections 2024-04-25T01:09:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77280678c0 0x7f7728069d80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77480d1b50 0x7f774815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77480b5580 0x7f7748144230 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 --2- 172.21.15.33:0/3043654903 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77480b50d0 0x7f774806d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 >> 172.21.15.33:0/3043654903 conn(0x7f774809a9d0 msgr2=0x7f774815d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 shutdown_connections 2024-04-25T01:09:27.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.848+0000 7f774e9c3640 1 -- 172.21.15.33:0/3043654903 wait complete. 2024-04-25T01:09:27.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.976+0000 7f6c2ed8f640 1 Processor -- start 2024-04-25T01:09:27.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2ed8f640 1 -- start start 2024-04-25T01:09:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2ed8f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6c280cd580 0x7f6c280b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2ed8f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c280b4840 con 0x7f6c2815ae70 2024-04-25T01:09:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2ed8f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c280d0100 con 0x7f6c2806dfe0 2024-04-25T01:09:27.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2ed8f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6c280b5410 con 0x7f6c280cd580 2024-04-25T01:09:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2d32c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6c2815ae70 0x7f6c2806e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:49158/0 (socket says 172.21.15.33:49158) 2024-04-25T01:09:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2d32c640 1 -- 172.21.15.33:0/2140485475 learned_addr learned my addr 172.21.15.33:0/2140485475 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2cb2b640 1 --2- 172.21.15.33:0/2140485475 >> v2:172.21.15.138:3300/0 conn(0x7f6c280cd580 0x7f6c280b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769051305 0 0) 0x7f6c280b4840 con 0x7f6c2815ae70 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6c10003530 con 0x7f6c2815ae70 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 550950203 0 0) 0x7f6c10003530 con 0x7f6c2815ae70 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 >> v2:172.21.15.138:3300/0 conn(0x7f6c280cd580 msgr2=0x7f6c280b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 --2- 172.21.15.33:0/2140485475 >> v2:172.21.15.138:3300/0 conn(0x7f6c280cd580 0x7f6c280b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 >> v1:172.21.15.138:6789/0 conn(0x7f6c2806dfe0 legacy=0x7f6c280cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c280b4ae0 con 0x7f6c2815ae70 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f6c20001a20 con 0x7f6c2815ae70 2024-04-25T01:09:27.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 >> v1:172.21.15.33:6789/0 conn(0x7f6c2815ae70 legacy=0x7f6c2806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c10003a10 0x7f6c10003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c10004810 0x7f6c10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c100051d0 0x7f6c1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280b5410 con 0x7f6c10004810 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1000c100 con 0x7f6c100051d0 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1000e510 con 0x7f6c10003a10 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c27fff640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c10004810 0x7f6c10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2d32c640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c100051d0 0x7f6c1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2cb2b640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c10003a10 0x7f6c10003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c27fff640 1 -- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c10003a10 msgr2=0x7f6c10003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c27fff640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c10003a10 0x7f6c10003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c27fff640 1 -- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c100051d0 msgr2=0x7f6c1000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:27.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c27fff640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c100051d0 0x7f6c1000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c27fff640 1 -- 172.21.15.33:0/2140485475 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c10003530 con 0x7f6c10004810 2024-04-25T01:09:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2d32c640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c100051d0 0x7f6c1000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2d32c640 1 -- 172.21.15.33:0/2140485475 reap_dead start 2024-04-25T01:09:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.980+0000 7f6c2cb2b640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c10003a10 0x7f6c10003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:27.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c27fff640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c10004810 0x7f6c10004c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c18001750 tx=0x7f6c1801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602127fcc10d6f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c180013a0 con 0x7f6c10004810 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c180052f0 con 0x7f6c10004810 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c277fe640 1 -- 172.21.15.33:0/2140485475 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18003840 con 0x7f6c10004810 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/2140485475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c10004810 msgr2=0x7f6c10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c10004810 0x7f6c10004c90 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6c18001750 tx=0x7f6c1801c3c0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/2140485475 shutdown_connections 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/2140485475 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c10004810 0x7f6c10004c90 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/2140485475 >> 172.21.15.33:0/2140485475 conn(0x7f6c280677a0 msgr2=0x7f6c2806c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/2140485475 shutdown_connections 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/2140485475 wait complete. 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 Processor -- start 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- start start 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c10004810 0x7f6c2815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c281e3e80 0x7f6c281590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c2814dde0 0x7f6c2814e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280cfc90 con 0x7f6c2814dde0 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6c280b50a0 con 0x7f6c10004810 2024-04-25T01:09:27.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2ed8f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6c280cfe10 con 0x7f6c281e3e80 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c2814dde0 0x7f6c2814e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c2814dde0 0x7f6c2814e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60518/0 (socket says 172.21.15.33:60518) 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 -- 172.21.15.33:0/3130025730 learned_addr learned my addr 172.21.15.33:0/3130025730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c27fff640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c281e3e80 0x7f6c281590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2cb2b640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c10004810 0x7f6c2815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 -- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c281e3e80 msgr2=0x7f6c281590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c281e3e80 0x7f6c281590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 -- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c10004810 msgr2=0x7f6c2815c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:27.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c10004810 0x7f6c2815c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.984+0000 7f6c2d32c640 1 -- 172.21.15.33:0/3130025730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c281efef0 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c2d32c640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c2814dde0 0x7f6c2814e220 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6c280cece0 tx=0x7f6c20010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c257fa640 1 -- 172.21.15.33:0/3130025730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2001a070 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c257fa640 1 -- 172.21.15.33:0/3130025730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c20011040 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2815cd20 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c257fa640 1 -- 172.21.15.33:0/3130025730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c2000ec80 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2815d2e0 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c28069040 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c257fa640 1 -- 172.21.15.33:0/3130025730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6c2001d070 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c257fa640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6c140678f0 0x7f6c14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c257fa640 1 -- 172.21.15.33:0/3130025730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6c200888f0 con 0x7f6c2814dde0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c2cb2b640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6c140678f0 0x7f6c14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:27.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.988+0000 7f6c2cb2b640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6c140678f0 0x7f6c14069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c28069dd0 tx=0x7f6c1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:27.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:27.992+0000 7f6c257fa640 1 -- 172.21.15.33:0/3130025730 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6c20058fa0 con 0x7f6c2814dde0 2024-04-25T01:09:28.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.144+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6c280d05b0 con 0x7f6c2814dde0 2024-04-25T01:09:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c257fa640 1 -- 172.21.15.33:0/3130025730 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f6c200588a0 con 0x7f6c2814dde0 2024-04-25T01:09:28.140 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6c140678f0 msgr2=0x7f6c14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6c140678f0 0x7f6c14069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c28069dd0 tx=0x7f6c1c008040 comp rx=0 tx=0).stop 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c2814dde0 msgr2=0x7f6c2814e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c2814dde0 0x7f6c2814e220 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6c280cece0 tx=0x7f6c20010d70 comp rx=0 tx=0).stop 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 shutdown_connections 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6c140678f0 0x7f6c14069db0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c2814dde0 0x7f6c2814e220 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6c281e3e80 0x7f6c281590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 --2- 172.21.15.33:0/3130025730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c10004810 0x7f6c2815c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 >> 172.21.15.33:0/3130025730 conn(0x7f6c280677a0 msgr2=0x7f6c280cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 shutdown_connections 2024-04-25T01:09:28.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.148+0000 7f6c2ed8f640 1 -- 172.21.15.33:0/3130025730 wait complete. 2024-04-25T01:09:28.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.280+0000 7f8fd40fe640 1 Processor -- start 2024-04-25T01:09:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.280+0000 7f8fd40fe640 1 -- start start 2024-04-25T01:09:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8fcc0d23c0 0x7f8fcc0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc161c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fcc0d29f0 con 0x7f8fcc161720 2024-04-25T01:09:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8fcc0ceae0 con 0x7f8fcc0d23c0 2024-04-25T01:09:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc0cedc0 con 0x7f8fcc0b3ba0 2024-04-25T01:09:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8fcc0d23c0 0x7f8fcc0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8fcc0d23c0 0x7f8fcc0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60532/0 (socket says 172.21.15.33:60532) 2024-04-25T01:09:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 -- 172.21.15.33:0/516420074 learned_addr learned my addr 172.21.15.33:0/516420074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd269b640 1 --2- 172.21.15.33:0/516420074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 -- 172.21.15.33:0/516420074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 msgr2=0x7f8fcc161c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 --2- 172.21.15.33:0/516420074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc161c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 -- 172.21.15.33:0/516420074 >> v1:172.21.15.138:6789/0 conn(0x7f8fcc161720 legacy=0x7f8fcc0cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 -- 172.21.15.33:0/516420074 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc0cf0a0 con 0x7f8fcc0d23c0 2024-04-25T01:09:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd1699640 1 --2- 172.21.15.33:0/516420074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc0d23c0 0x7f8fcc0b3660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009870 tx=0x7f8fbc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cb9841fdc51431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd0e98640 1 -- 172.21.15.33:0/516420074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0b2070 con 0x7f8fcc0d23c0 2024-04-25T01:09:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd0e98640 1 -- 172.21.15.33:0/516420074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0a9070 con 0x7f8fcc0d23c0 2024-04-25T01:09:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd0e98640 1 -- 172.21.15.33:0/516420074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0ad880 con 0x7f8fcc0d23c0 2024-04-25T01:09:28.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 -- 172.21.15.33:0/516420074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc0d23c0 msgr2=0x7f8fcc0b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/516420074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc0d23c0 0x7f8fcc0b3660 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc009870 tx=0x7f8fbc01c560 comp rx=0 tx=0).stop 2024-04-25T01:09:28.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 -- 172.21.15.33:0/516420074 shutdown_connections 2024-04-25T01:09:28.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/516420074 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc161c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/516420074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc0d23c0 0x7f8fcc0b3660 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 -- 172.21.15.33:0/516420074 >> 172.21.15.33:0/516420074 conn(0x7f8fcc0d3e30 msgr2=0x7f8fcc0d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:28.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.284+0000 7f8fd40fe640 1 -- 172.21.15.33:0/516420074 shutdown_connections 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- 172.21.15.33:0/516420074 wait complete. 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 Processor -- start 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- start start 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fcc0d23c0 0x7f8fcc0c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc161720 0x7f8fcc157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc16b0d0 con 0x7f8fcc161720 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fcc16af50 con 0x7f8fcc0d23c0 2024-04-25T01:09:28.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8fcc16b250 con 0x7f8fcc0b3ba0 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc161720 0x7f8fcc157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc161720 0x7f8fcc157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60542/0 (socket says 172.21.15.33:60542) 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 -- 172.21.15.33:0/3537222494 learned_addr learned my addr 172.21.15.33:0/3537222494 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd1699640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fcc0d23c0 0x7f8fcc0c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd1e9a640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 -- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 msgr2=0x7f8fcc0ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc0ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 -- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fcc0d23c0 msgr2=0x7f8fcc0c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fcc0d23c0 0x7f8fcc0c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 -- 172.21.15.33:0/3537222494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fcc0b0c90 con 0x7f8fcc161720 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd1e9a640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd269b640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc161720 0x7f8fcc157ee0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d560 tx=0x7f8fc400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fc27fc640 1 -- 172.21.15.33:0/3537222494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40125f0 con 0x7f8fcc161720 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fc27fc640 1 -- 172.21.15.33:0/3537222494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fc4019070 con 0x7f8fcc161720 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fc27fc640 1 -- 172.21.15.33:0/3537222494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc40129f0 con 0x7f8fcc161720 2024-04-25T01:09:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fcc0b0f70 con 0x7f8fcc161720 2024-04-25T01:09:28.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fcc0c8850 con 0x7f8fcc161720 2024-04-25T01:09:28.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.288+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fcc069040 con 0x7f8fcc161720 2024-04-25T01:09:28.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.292+0000 7f8fc27fc640 1 -- 172.21.15.33:0/3537222494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8fc4012b90 con 0x7f8fcc161720 2024-04-25T01:09:28.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.292+0000 7f8fc27fc640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8fb00678c0 0x7f8fb0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.292+0000 7f8fc27fc640 1 -- 172.21.15.33:0/3537222494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8fc4086490 con 0x7f8fcc161720 2024-04-25T01:09:28.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.292+0000 7f8fd1e9a640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8fb00678c0 0x7f8fb0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.292+0000 7f8fd1e9a640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8fb00678c0 0x7f8fb0069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8007fd0 tx=0x7f8fc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.296+0000 7f8fc27fc640 1 -- 172.21.15.33:0/3537222494 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8fc4056bd0 con 0x7f8fcc161720 2024-04-25T01:09:28.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.460+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8fcc0b16e0 con 0x7f8fcc161720 2024-04-25T01:09:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fc27fc640 1 -- 172.21.15.33:0/3537222494 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f8fc40564d0 con 0x7f8fcc161720 2024-04-25T01:09:28.455 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8fb00678c0 msgr2=0x7f8fb0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8fb00678c0 0x7f8fb0069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8007fd0 tx=0x7f8fc8009040 comp rx=0 tx=0).stop 2024-04-25T01:09:28.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc161720 msgr2=0x7f8fcc157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc161720 0x7f8fcc157ee0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d560 tx=0x7f8fc400da20 comp rx=0 tx=0).stop 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 shutdown_connections 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8fb00678c0 0x7f8fb0069d80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8fcc161720 0x7f8fcc157ee0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fcc0d23c0 0x7f8fcc0c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 --2- 172.21.15.33:0/3537222494 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8fcc0b3ba0 0x7f8fcc0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 >> 172.21.15.33:0/3537222494 conn(0x7f8fcc0d3e30 msgr2=0x7f8fcc06bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 shutdown_connections 2024-04-25T01:09:28.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.464+0000 7f8fd40fe640 1 -- 172.21.15.33:0/3537222494 wait complete. 2024-04-25T01:09:28.474 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d4de65640 1 Processor -- start 2024-04-25T01:09:28.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d4de65640 1 -- start start 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d4de65640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5d481444f0 0x7f5d480b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d4de65640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5d48072540 0x7f5d480ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d4de65640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d481448d0 con 0x7f5d4815e660 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d4de65640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f5d480ce0c0 con 0x7f5d481444f0 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d4de65640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f5d480b5b60 con 0x7f5d48072540 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5d481444f0 0x7f5d480b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5d481444f0 0x7f5d480b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34594/0 (socket says 172.21.15.33:34594) 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 -- 172.21.15.33:0/1422899411 learned_addr learned my addr 172.21.15.33:0/1422899411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 -- 172.21.15.33:0/1422899411 >> v2:172.21.15.138:3300/0 conn(0x7f5d48072540 msgr2=0x7f5d480ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 --2- 172.21.15.33:0/1422899411 >> v2:172.21.15.138:3300/0 conn(0x7f5d48072540 0x7f5d480ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 -- 172.21.15.33:0/1422899411 >> v1:172.21.15.138:6789/0 conn(0x7f5d4815e660 legacy=0x7f5d48071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:28.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 -- 172.21.15.33:0/1422899411 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4815ec20 con 0x7f5d481444f0 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.596+0000 7f5d477fe640 1 --2- 172.21.15.33:0/1422899411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481444f0 0x7f5d480b5620 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5d480b2ba0 tx=0x7f5d3801c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dd602912357a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d467fc640 1 -- 172.21.15.33:0/1422899411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380b0070 con 0x7f5d481444f0 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d467fc640 1 -- 172.21.15.33:0/1422899411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d380ab600 con 0x7f5d481444f0 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d467fc640 1 -- 172.21.15.33:0/1422899411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d380ab9d0 con 0x7f5d481444f0 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- 172.21.15.33:0/1422899411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481444f0 msgr2=0x7f5d480b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 --2- 172.21.15.33:0/1422899411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481444f0 0x7f5d480b5620 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5d480b2ba0 tx=0x7f5d3801c290 comp rx=0 tx=0).stop 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- 172.21.15.33:0/1422899411 shutdown_connections 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 --2- 172.21.15.33:0/1422899411 >> v2:172.21.15.138:3300/0 conn(0x7f5d48072540 0x7f5d480ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 --2- 172.21.15.33:0/1422899411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d481444f0 0x7f5d480b5620 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- 172.21.15.33:0/1422899411 >> 172.21.15.33:0/1422899411 conn(0x7f5d480b1810 msgr2=0x7f5d480b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:28.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- 172.21.15.33:0/1422899411 shutdown_connections 2024-04-25T01:09:28.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- 172.21.15.33:0/1422899411 wait complete. 2024-04-25T01:09:28.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 Processor -- start 2024-04-25T01:09:28.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- start start 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d48072540 0x7f5d4813a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d481444f0 0x7f5d481427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4815e660 0x7f5d4815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d480b37a0 con 0x7f5d4815e660 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d480b3620 con 0x7f5d48072540 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d4de65640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5d48167a00 con 0x7f5d481444f0 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4815e660 0x7f5d4815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d47fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4815e660 0x7f5d4815b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34598/0 (socket says 172.21.15.33:34598) 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d47fff640 1 -- 172.21.15.33:0/2680786034 learned_addr learned my addr 172.21.15.33:0/2680786034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d477fe640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d48072540 0x7f5d4813a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d46ffd640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d481444f0 0x7f5d481427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d47fff640 1 -- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d481444f0 msgr2=0x7f5d481427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d47fff640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d481444f0 0x7f5d481427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d47fff640 1 -- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d48072540 msgr2=0x7f5d4813a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.600+0000 7f5d47fff640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d48072540 0x7f5d4813a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d47fff640 1 -- 172.21.15.33:0/2680786034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d4813a830 con 0x7f5d4815e660 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d47fff640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4815e660 0x7f5d4815b0e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000b9e0 tx=0x7f5d4000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d2ffff640 1 -- 172.21.15.33:0/2680786034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40019070 con 0x7f5d4815e660 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d2ffff640 1 -- 172.21.15.33:0/2680786034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d4000cb20 con 0x7f5d4815e660 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d48141760 con 0x7f5d4815e660 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d2ffff640 1 -- 172.21.15.33:0/2680786034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40014440 con 0x7f5d4815e660 2024-04-25T01:09:28.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d48141ce0 con 0x7f5d4815e660 2024-04-25T01:09:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d48069040 con 0x7f5d4815e660 2024-04-25T01:09:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d2ffff640 1 -- 172.21.15.33:0/2680786034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5d4001c070 con 0x7f5d4815e660 2024-04-25T01:09:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d2ffff640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5d340678c0 0x7f5d34069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d2ffff640 1 -- 172.21.15.33:0/2680786034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5d40085e10 con 0x7f5d4815e660 2024-04-25T01:09:28.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d477fe640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5d340678c0 0x7f5d34069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.604+0000 7f5d477fe640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5d340678c0 0x7f5d34069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5d380ad000 tx=0x7f5d38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.608+0000 7f5d2ffff640 1 -- 172.21.15.33:0/2680786034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5d400564c0 con 0x7f5d4815e660 2024-04-25T01:09:28.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.776+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d48133d40 con 0x7f5d4815e660 2024-04-25T01:09:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.776+0000 7f5d2ffff640 1 -- 172.21.15.33:0/2680786034 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f5d40055dc0 con 0x7f5d4815e660 2024-04-25T01:09:28.770 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5d340678c0 msgr2=0x7f5d34069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5d340678c0 0x7f5d34069d80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5d380ad000 tx=0x7f5d38008040 comp rx=0 tx=0).stop 2024-04-25T01:09:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4815e660 msgr2=0x7f5d4815b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4815e660 0x7f5d4815b0e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000b9e0 tx=0x7f5d4000bea0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 shutdown_connections 2024-04-25T01:09:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f5d340678c0 0x7f5d34069d80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d4815e660 0x7f5d4815b0e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5d481444f0 0x7f5d481427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 --2- 172.21.15.33:0/2680786034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d48072540 0x7f5d4813a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 >> 172.21.15.33:0/2680786034 conn(0x7f5d480b1810 msgr2=0x7f5d4806cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 shutdown_connections 2024-04-25T01:09:28.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.780+0000 7f5d4de65640 1 -- 172.21.15.33:0/2680786034 wait complete. 2024-04-25T01:09:28.787 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:09:28.787 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:09:28.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:28.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.924+0000 7faa9f5fd640 1 Processor -- start 2024-04-25T01:09:28.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.924+0000 7faa9f5fd640 1 -- start start 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9f5fd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7faa980d03c0 0x7faa980aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9f5fd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980add80 con 0x7faa9815bf80 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9f5fd640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa980d00e0 con 0x7faa9806b210 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9f5fd640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7faa980cfe90 con 0x7faa980d03c0 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faa9815bf80 0x7faa980b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39144/0 (socket says 172.21.15.33:39144) 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 -- 172.21.15.33:0/190323850 learned_addr learned my addr 172.21.15.33:0/190323850 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9db9a640 1 --2- 172.21.15.33:0/190323850 >> v2:172.21.15.138:3300/0 conn(0x7faa980d03c0 0x7faa980aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1094944746 0 0) 0x7faa980add80 con 0x7faa9815bf80 2024-04-25T01:09:28.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa80003530 con 0x7faa9815bf80 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 670086368 0 0) 0x7faa80003530 con 0x7faa9815bf80 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 >> v2:172.21.15.138:3300/0 conn(0x7faa980d03c0 msgr2=0x7faa980aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 --2- 172.21.15.33:0/190323850 >> v2:172.21.15.138:3300/0 conn(0x7faa980d03c0 0x7faa980aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 >> v1:172.21.15.138:6789/0 conn(0x7faa9806b210 legacy=0x7faa9806b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa980b3f10 con 0x7faa9815bf80 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7faa8c001680 con 0x7faa9815bf80 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 >> v1:172.21.15.33:6789/0 conn(0x7faa9815bf80 legacy=0x7faa980b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80003a10 0x7faa80003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80004810 0x7faa80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa800051d0 0x7faa8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa980cfe90 con 0x7faa80004810 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa8000c0d0 con 0x7faa800051d0 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa8000c2d0 con 0x7faa80003a10 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80004810 0x7faa80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 -- 172.21.15.33:0/190323850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80003a10 msgr2=0x7faa80003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80003a10 0x7faa80003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 -- 172.21.15.33:0/190323850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa800051d0 msgr2=0x7faa8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa800051d0 0x7faa8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9db9a640 1 -- 172.21.15.33:0/190323850 reap_dead start 2024-04-25T01:09:28.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 -- 172.21.15.33:0/190323850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa80003530 con 0x7faa80004810 2024-04-25T01:09:28.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.928+0000 7faa9cb98640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80004810 0x7faa80004c90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c190 tx=0x7faa8c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890a1c7e49830e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c007c70 con 0x7faa80004810 2024-04-25T01:09:28.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c005ae0 con 0x7faa80004810 2024-04-25T01:09:28.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa97fff640 1 -- 172.21.15.33:0/190323850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c007070 con 0x7faa80004810 2024-04-25T01:09:28.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- 172.21.15.33:0/190323850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80004810 msgr2=0x7faa80004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80004810 0x7faa80004c90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00c190 tx=0x7faa8c01f330 comp rx=0 tx=0).stop 2024-04-25T01:09:28.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- 172.21.15.33:0/190323850 shutdown_connections 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/190323850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa80004810 0x7faa80004c90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- 172.21.15.33:0/190323850 >> 172.21.15.33:0/190323850 conn(0x7faa980b29c0 msgr2=0x7faa980b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- 172.21.15.33:0/190323850 shutdown_connections 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- 172.21.15.33:0/190323850 wait complete. 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 Processor -- start 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- start start 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80004810 0x7faa981e25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa981e4c30 0x7faa981e3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa981e3d70 0x7faa981e4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b44d0 con 0x7faa981e3d70 2024-04-25T01:09:28.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faa980b3cc0 con 0x7faa981e4c30 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faa980aefa0 con 0x7faa80004810 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa981e3d70 0x7faa981e4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa981e3d70 0x7faa981e4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34606/0 (socket says 172.21.15.33:34606) 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 -- 172.21.15.33:0/2673711320 learned_addr learned my addr 172.21.15.33:0/2673711320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9cb98640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa981e4c30 0x7faa981e3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9d399640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80004810 0x7faa981e25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 -- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80004810 msgr2=0x7faa981e25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80004810 0x7faa981e25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 -- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa981e4c30 msgr2=0x7faa981e3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa981e4c30 0x7faa981e3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 -- 172.21.15.33:0/2673711320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa981e2ea0 con 0x7faa981e3d70 2024-04-25T01:09:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9db9a640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa981e3d70 0x7faa981e4200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa980b0900 tx=0x7faa8800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa95ffb640 1 -- 172.21.15.33:0/2673711320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88017020 con 0x7faa981e3d70 2024-04-25T01:09:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa95ffb640 1 -- 172.21.15.33:0/2673711320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa88004030 con 0x7faa981e3d70 2024-04-25T01:09:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa981e3180 con 0x7faa981e3d70 2024-04-25T01:09:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa95ffb640 1 -- 172.21.15.33:0/2673711320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa88004400 con 0x7faa981e3d70 2024-04-25T01:09:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.932+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa981e6a90 con 0x7faa981e3d70 2024-04-25T01:09:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.936+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa98069040 con 0x7faa981e3d70 2024-04-25T01:09:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.936+0000 7faa95ffb640 1 -- 172.21.15.33:0/2673711320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa8801a020 con 0x7faa981e3d70 2024-04-25T01:09:28.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.936+0000 7faa95ffb640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faa840678f0 0x7faa84069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:28.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.936+0000 7faa95ffb640 1 -- 172.21.15.33:0/2673711320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7faa880861f0 con 0x7faa981e3d70 2024-04-25T01:09:28.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.936+0000 7faa9d399640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faa840678f0 0x7faa84069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:28.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.936+0000 7faa9d399640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faa840678f0 0x7faa84069db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa980cf200 tx=0x7faa90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:28.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:28.940+0000 7faa95ffb640 1 -- 172.21.15.33:0/2673711320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faa88021020 con 0x7faa981e3d70 2024-04-25T01:09:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.100+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa9806c050 con 0x7faa981e3d70 2024-04-25T01:09:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa95ffb640 1 -- 172.21.15.33:0/2673711320 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7faa880569c0 con 0x7faa981e3d70 2024-04-25T01:09:29.095 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:29.096 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:29.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faa840678f0 msgr2=0x7faa84069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faa840678f0 0x7faa84069db0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faa980cf200 tx=0x7faa90009040 comp rx=0 tx=0).stop 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa981e3d70 msgr2=0x7faa981e4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa981e3d70 0x7faa981e4200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faa980b0900 tx=0x7faa8800ec00 comp rx=0 tx=0).stop 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 shutdown_connections 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faa840678f0 0x7faa84069db0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa981e3d70 0x7faa981e4200 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa981e4c30 0x7faa981e3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 --2- 172.21.15.33:0/2673711320 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faa80004810 0x7faa981e25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 >> 172.21.15.33:0/2673711320 conn(0x7faa980b29c0 msgr2=0x7faa98158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 shutdown_connections 2024-04-25T01:09:29.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.104+0000 7faa9f5fd640 1 -- 172.21.15.33:0/2673711320 wait complete. 2024-04-25T01:09:29.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:29.116 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab89e69640 1 Processor -- start 2024-04-25T01:09:29.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab89e69640 1 -- start start 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab89e69640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fab841587b0 0x7fab8406c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab89e69640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fab8406c760 0x7fab840ae430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab89e69640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab840b4000 con 0x7fab841583d0 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab89e69640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fab840b4c10 con 0x7fab8406c760 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab89e69640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fab840b42e0 con 0x7fab841587b0 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab837fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab841583d0 0x7fab840720f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39160/0 (socket says 172.21.15.33:39160) 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab837fe640 1 -- 172.21.15.33:0/1004779586 learned_addr learned my addr 172.21.15.33:0/1004779586 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab83fff640 1 --2- 172.21.15.33:0/1004779586 >> v2:172.21.15.33:3300/0 conn(0x7fab8406c760 0x7fab840ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab82ffd640 1 --2- 172.21.15.33:0/1004779586 >> v2:172.21.15.138:3300/0 conn(0x7fab841587b0 0x7fab8406c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab827fc640 1 -- 172.21.15.33:0/1004779586 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106461081 0 0) 0x7fab840b4000 con 0x7fab841583d0 2024-04-25T01:09:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab827fc640 1 -- 172.21.15.33:0/1004779586 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab6c003530 con 0x7fab841583d0 2024-04-25T01:09:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab83fff640 1 -- 172.21.15.33:0/1004779586 >> v2:172.21.15.138:3300/0 conn(0x7fab841587b0 msgr2=0x7fab8406c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab83fff640 1 --2- 172.21.15.33:0/1004779586 >> v2:172.21.15.138:3300/0 conn(0x7fab841587b0 0x7fab8406c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab827fc640 1 -- 172.21.15.33:0/1004779586 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2295109507 0 0) 0x7fab6c003530 con 0x7fab841583d0 2024-04-25T01:09:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab83fff640 1 -- 172.21.15.33:0/1004779586 >> v1:172.21.15.33:6789/0 conn(0x7fab841583d0 legacy=0x7fab840720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.252+0000 7fab83fff640 1 -- 172.21.15.33:0/1004779586 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab840b45c0 con 0x7fab8406c760 2024-04-25T01:09:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab83fff640 1 --2- 172.21.15.33:0/1004779586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab840ae430 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ada069db47ac24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab827fc640 1 -- 172.21.15.33:0/1004779586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c0b2070 con 0x7fab8406c760 2024-04-25T01:09:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab827fc640 1 -- 172.21.15.33:0/1004779586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7c0a55e0 con 0x7fab8406c760 2024-04-25T01:09:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab827fc640 1 -- 172.21.15.33:0/1004779586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74001680 con 0x7fab8406c760 2024-04-25T01:09:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 -- 172.21.15.33:0/1004779586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 msgr2=0x7fab840ae430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 --2- 172.21.15.33:0/1004779586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab840ae430 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009d90 tx=0x7fab7c01c850 comp rx=0 tx=0).stop 2024-04-25T01:09:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 -- 172.21.15.33:0/1004779586 shutdown_connections 2024-04-25T01:09:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 --2- 172.21.15.33:0/1004779586 >> v2:172.21.15.138:3300/0 conn(0x7fab841587b0 0x7fab8406c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 --2- 172.21.15.33:0/1004779586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab840ae430 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 -- 172.21.15.33:0/1004779586 >> 172.21.15.33:0/1004779586 conn(0x7fab840715d0 msgr2=0x7fab840719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 -- 172.21.15.33:0/1004779586 shutdown_connections 2024-04-25T01:09:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 -- 172.21.15.33:0/1004779586 wait complete. 2024-04-25T01:09:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.256+0000 7fab89e69640 1 Processor -- start 2024-04-25T01:09:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 -- start start 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab84148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab841583d0 0x7fab84142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab841587b0 0x7fab8414a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab840af300 con 0x7fab8406c760 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab840af180 con 0x7fab841583d0 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fab840af480 con 0x7fab841587b0 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab84148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab84148050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34632/0 (socket says 172.21.15.33:34632) 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 -- 172.21.15.33:0/4277042064 learned_addr learned my addr 172.21.15.33:0/4277042064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:29.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab82ffd640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab841583d0 0x7fab84142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 -- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab841587b0 msgr2=0x7fab8414a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab841587b0 0x7fab8414a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 -- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab841583d0 msgr2=0x7fab84142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab841583d0 0x7fab84142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 -- 172.21.15.33:0/4277042064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8414b1b0 con 0x7fab8406c760 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab837fe640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab84148050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c750 tx=0x7fab7400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab6bfff640 1 -- 172.21.15.33:0/4277042064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74005ae0 con 0x7fab8406c760 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab6bfff640 1 -- 172.21.15.33:0/4277042064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab74005ca0 con 0x7fab8406c760 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8414b490 con 0x7fab8406c760 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab6bfff640 1 -- 172.21.15.33:0/4277042064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740072b0 con 0x7fab8406c760 2024-04-25T01:09:29.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8414bd30 con 0x7fab8406c760 2024-04-25T01:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab84069040 con 0x7fab8406c760 2024-04-25T01:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab6bfff640 1 -- 172.21.15.33:0/4277042064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab74007450 con 0x7fab8406c760 2024-04-25T01:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab6bfff640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab700678c0 0x7fab70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab6bfff640 1 -- 172.21.15.33:0/4277042064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fab74086030 con 0x7fab8406c760 2024-04-25T01:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.260+0000 7fab82ffd640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab700678c0 0x7fab70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.264+0000 7fab82ffd640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab700678c0 0x7fab70069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab84143cb0 tx=0x7fab78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.264+0000 7fab6bfff640 1 -- 172.21.15.33:0/4277042064 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab74056770 con 0x7fab8406c760 2024-04-25T01:09:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.428+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fab8406bf00 con 0x7fab8406c760 2024-04-25T01:09:29.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab6bfff640 1 -- 172.21.15.33:0/4277042064 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fab74056070 con 0x7fab8406c760 2024-04-25T01:09:29.424 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab700678c0 msgr2=0x7fab70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab700678c0 0x7fab70069d80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fab84143cb0 tx=0x7fab78008040 comp rx=0 tx=0).stop 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 msgr2=0x7fab84148050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab84148050 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c750 tx=0x7fab7400cc10 comp rx=0 tx=0).stop 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 shutdown_connections 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab700678c0 0x7fab70069d80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab841587b0 0x7fab8414a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab841583d0 0x7fab84142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 --2- 172.21.15.33:0/4277042064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab8406c760 0x7fab84148050 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 >> 172.21.15.33:0/4277042064 conn(0x7fab840715d0 msgr2=0x7fab840cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 shutdown_connections 2024-04-25T01:09:29.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.432+0000 7fab89e69640 1 -- 172.21.15.33:0/4277042064 wait complete. 2024-04-25T01:09:29.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:29.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa35a37c640 1 Processor -- start 2024-04-25T01:09:29.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa35a37c640 1 -- start start 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa35a37c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 0x7fa354166c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa35a37c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa354160780 con 0x7fa35406eb60 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa35a37c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3540cee10 con 0x7fa354145660 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa35a37c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa354145a40 con 0x7fa354072180 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa3537fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa35406eb60 0x7fa354160070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39170/0 (socket says 172.21.15.33:39170) 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa3537fe640 1 -- 172.21.15.33:0/2508129674 learned_addr learned my addr 172.21.15.33:0/2508129674 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa358919640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 0x7fa354166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2624745095 0 0) 0x7fa354160780 con 0x7fa35406eb60 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa33c003530 con 0x7fa35406eb60 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2610240024 0 0) 0x7fa33c003530 con 0x7fa35406eb60 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 msgr2=0x7fa354166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 0x7fa354166c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 >> v1:172.21.15.138:6789/0 conn(0x7fa354145660 legacy=0x7fa35415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3540cf0f0 con 0x7fa35406eb60 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fa348001680 con 0x7fa35406eb60 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 >> v1:172.21.15.33:6789/0 conn(0x7fa35406eb60 legacy=0x7fa354160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 0x7fa33c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa33c003f70 0x7fa33c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c00ba80 con 0x7fa33c004d50 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c00bc30 con 0x7fa33c003f70 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c00be30 con 0x7fa354072180 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa358919640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa353fff640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 0x7fa33c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa358919640 1 -- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 msgr2=0x7fa33c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa358919640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354072180 0x7fa33c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa3537fe640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa33c003f70 0x7fa33c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa358919640 1 -- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa33c003f70 msgr2=0x7fa33c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa358919640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa33c003f70 0x7fa33c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.564+0000 7fa358919640 1 -- 172.21.15.33:0/2508129674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c003530 con 0x7fa33c004d50 2024-04-25T01:09:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa358919640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009f90 tx=0x7fa34c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1428a5692064e1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0b2070 con 0x7fa33c004d50 2024-04-25T01:09:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c0a5c50 con 0x7fa33c004d50 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa352ffd640 1 -- 172.21.15.33:0/2508129674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c009c10 con 0x7fa33c004d50 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- 172.21.15.33:0/2508129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c004d50 msgr2=0x7fa33c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 --2- 172.21.15.33:0/2508129674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa33c004d50 0x7fa33c0051d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009f90 tx=0x7fa34c01cc40 comp rx=0 tx=0).stop 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa358919640 1 -- 172.21.15.33:0/2508129674 reap_dead start 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- 172.21.15.33:0/2508129674 shutdown_connections 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- 172.21.15.33:0/2508129674 >> 172.21.15.33:0/2508129674 conn(0x7fa3540b1810 msgr2=0x7fa3540b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- 172.21.15.33:0/2508129674 shutdown_connections 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- 172.21.15.33:0/2508129674 wait complete. 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 Processor -- start 2024-04-25T01:09:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- start start 2024-04-25T01:09:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354140eb0 0x7fa35415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa354158fc0 0x7fa3540ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35415b920 0x7fa35415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540cf6b0 con 0x7fa35415b920 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa354070ce0 con 0x7fa354158fc0 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa35a37c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa35406efa0 con 0x7fa354140eb0 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.568+0000 7fa358919640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35415b920 0x7fa35415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35415b920 0x7fa35415bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34648/0 (socket says 172.21.15.33:34648) 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 -- 172.21.15.33:0/4071761543 learned_addr learned my addr 172.21.15.33:0/4071761543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa353fff640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354140eb0 0x7fa35415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa3537fe640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa354158fc0 0x7fa3540ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 -- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354140eb0 msgr2=0x7fa35415a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354140eb0 0x7fa35415a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 -- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa354158fc0 msgr2=0x7fa3540ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa354158fc0 0x7fa3540ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 -- 172.21.15.33:0/4071761543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3541594c0 con 0x7fa35415b920 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa358919640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35415b920 0x7fa35415bdb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0b0040 tx=0x7fa34c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa350ff9640 1 -- 172.21.15.33:0/4071761543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0b2040 con 0x7fa35415b920 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa350ff9640 1 -- 172.21.15.33:0/4071761543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c0bc070 con 0x7fa35415b920 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3541597a0 con 0x7fa35415b920 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa350ff9640 1 -- 172.21.15.33:0/4071761543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c009c10 con 0x7fa35415b920 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa354159d60 con 0x7fa35415b920 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa354069040 con 0x7fa35415b920 2024-04-25T01:09:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa350ff9640 1 -- 172.21.15.33:0/4071761543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa34c0a5a50 con 0x7fa35415b920 2024-04-25T01:09:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa350ff9640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa3400678f0 0x7fa340069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa350ff9640 1 -- 172.21.15.33:0/4071761543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa34c1207a0 con 0x7fa35415b920 2024-04-25T01:09:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa353fff640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa3400678f0 0x7fa340069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.572+0000 7fa353fff640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa3400678f0 0x7fa340069db0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c10 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.576+0000 7fa350ff9640 1 -- 172.21.15.33:0/4071761543 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa34c016950 con 0x7fa35415b920 2024-04-25T01:09:29.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.732+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa354072ac0 con 0x7fa35415b920 2024-04-25T01:09:29.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.732+0000 7fa350ff9640 1 -- 172.21.15.33:0/4071761543 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fa34c0f0f70 con 0x7fa35415b920 2024-04-25T01:09:29.725 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.732+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa3400678f0 msgr2=0x7fa340069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.732+0000 7fa35a37c640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa3400678f0 0x7fa340069db0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c10 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-04-25T01:09:29.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.732+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35415b920 msgr2=0x7fa35415bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.732+0000 7fa35a37c640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35415b920 0x7fa35415bdb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0b0040 tx=0x7fa34c009750 comp rx=0 tx=0).stop 2024-04-25T01:09:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 shutdown_connections 2024-04-25T01:09:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa3400678f0 0x7fa340069db0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa35415b920 0x7fa35415bdb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa354158fc0 0x7fa3540ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 --2- 172.21.15.33:0/4071761543 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa354140eb0 0x7fa35415a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 >> 172.21.15.33:0/4071761543 conn(0x7fa3540b1810 msgr2=0x7fa35406dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 shutdown_connections 2024-04-25T01:09:29.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.736+0000 7fa35a37c640 1 -- 172.21.15.33:0/4071761543 wait complete. 2024-04-25T01:09:29.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e32f0640 1 Processor -- start 2024-04-25T01:09:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e32f0640 1 -- start start 2024-04-25T01:09:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e32f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc15e0d0 0x7f77dc0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e32f0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77dc0d2ac0 0x7f77dc167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e32f0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77dc0b6cc0 con 0x7f77dc0d1f60 2024-04-25T01:09:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e32f0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f77dc0b7580 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e32f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f77dc0b6f30 con 0x7f77dc15e0d0 2024-04-25T01:09:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e188d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f77dc0d2ac0 0x7f77dc167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e108c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc15e0d0 0x7f77dc0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e088b640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f77dc0d1f60 0x7f77dc0d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:50920/0 (socket says 172.21.15.33:50920) 2024-04-25T01:09:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e088b640 1 -- 172.21.15.33:0/3933711451 learned_addr learned my addr 172.21.15.33:0/3933711451 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:29.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e188d640 1 -- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc15e0d0 msgr2=0x7f77dc0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e188d640 1 --2- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc15e0d0 0x7f77dc0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e188d640 1 -- 172.21.15.33:0/3933711451 >> v1:172.21.15.138:6789/0 conn(0x7f77dc0d1f60 legacy=0x7f77dc0d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e188d640 1 -- 172.21.15.33:0/3933711451 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc0b7210 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.876+0000 7f77e108c640 1 --2- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc15e0d0 0x7f77dc0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:09:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e188d640 1 --2- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc167cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009d90 tx=0x7f77d401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6859bdba9608fab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77cffff640 1 -- 172.21.15.33:0/3933711451 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40b2070 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77cffff640 1 -- 172.21.15.33:0/3933711451 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d40a5960 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77cffff640 1 -- 172.21.15.33:0/3933711451 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d40a5d30 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 -- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc0d2ac0 msgr2=0x7f77dc167cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 --2- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc167cf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009d90 tx=0x7f77d401c970 comp rx=0 tx=0).stop 2024-04-25T01:09:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 -- 172.21.15.33:0/3933711451 shutdown_connections 2024-04-25T01:09:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 --2- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc167cf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 --2- 172.21.15.33:0/3933711451 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc15e0d0 0x7f77dc0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 -- 172.21.15.33:0/3933711451 >> 172.21.15.33:0/3933711451 conn(0x7f77dc0d3dd0 msgr2=0x7f77dc0d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 -- 172.21.15.33:0/3933711451 shutdown_connections 2024-04-25T01:09:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 -- 172.21.15.33:0/3933711451 wait complete. 2024-04-25T01:09:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 Processor -- start 2024-04-25T01:09:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.880+0000 7f77e32f0640 1 -- start start 2024-04-25T01:09:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc0d1f60 0x7f77dc0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc15e0d0 0x7f77dc155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc16abd0 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc16aa50 con 0x7f77dc15e0d0 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f77dc16ad50 con 0x7f77dc0d1f60 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34658/0 (socket says 172.21.15.33:34658) 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 -- 172.21.15.33:0/2765707248 learned_addr learned my addr 172.21.15.33:0/2765707248 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e188d640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc15e0d0 0x7f77dc155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e108c640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc0d1f60 0x7f77dc0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 -- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc0d1f60 msgr2=0x7f77dc0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc0d1f60 0x7f77dc0cd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 -- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc15e0d0 msgr2=0x7f77dc155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc15e0d0 0x7f77dc155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 -- 172.21.15.33:0/2765707248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc0cdee0 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e188d640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc15e0d0 0x7f77dc155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e088b640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc155370 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c430 tx=0x7f77d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77cdffb640 1 -- 172.21.15.33:0/2765707248 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0017070 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77cdffb640 1 -- 172.21.15.33:0/2765707248 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d0005cf0 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc0c90d0 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77cdffb640 1 -- 172.21.15.33:0/2765707248 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00072f0 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc0c9690 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc069040 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77cdffb640 1 -- 172.21.15.33:0/2765707248 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f77d001a070 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77cdffb640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77c00678c0 0x7f77c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77cdffb640 1 -- 172.21.15.33:0/2765707248 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f77d0085460 con 0x7f77dc0d2ac0 2024-04-25T01:09:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.884+0000 7f77e108c640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77c00678c0 0x7f77c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:29.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.888+0000 7f77e108c640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77c00678c0 0x7f77c0069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0d4350 tx=0x7f77d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:29.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:29.888+0000 7f77cdffb640 1 -- 172.21.15.33:0/2765707248 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f77d0055ba0 con 0x7f77dc0d2ac0 2024-04-25T01:09:30.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.044+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77dc0b9770 con 0x7f77dc0d2ac0 2024-04-25T01:09:30.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.044+0000 7f77cdffb640 1 -- 172.21.15.33:0/2765707248 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f77d00554a0 con 0x7f77dc0d2ac0 2024-04-25T01:09:30.038 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:30.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77c00678c0 msgr2=0x7f77c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77c00678c0 0x7f77c0069d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f77dc0d4350 tx=0x7f77d8008040 comp rx=0 tx=0).stop 2024-04-25T01:09:30.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77dc0d2ac0 msgr2=0x7f77dc155370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc155370 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c430 tx=0x7f77d000c8f0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 shutdown_connections 2024-04-25T01:09:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f77c00678c0 0x7f77c0069d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77dc15e0d0 0x7f77dc155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77dc0d2ac0 0x7f77dc155370 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 --2- 172.21.15.33:0/2765707248 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f77dc0d1f60 0x7f77dc0cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 >> 172.21.15.33:0/2765707248 conn(0x7f77dc0d3dd0 msgr2=0x7f77dc06d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 shutdown_connections 2024-04-25T01:09:30.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.048+0000 7f77e32f0640 1 -- 172.21.15.33:0/2765707248 wait complete. 2024-04-25T01:09:30.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:30.114 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:09:30.116 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:09:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.180+0000 7faf4f585640 1 Processor -- start 2024-04-25T01:09:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.180+0000 7faf4f585640 1 -- start start 2024-04-25T01:09:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4f585640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 0x7faf480cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4f585640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf480ce280 con 0x7faf480ced40 2024-04-25T01:09:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4f585640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faf480d0760 con 0x7faf4815c020 2024-04-25T01:09:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4f585640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faf480d0530 con 0x7faf480aff10 2024-04-25T01:09:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4cb20640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7faf480ced40 0x7faf480cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39176/0 (socket says 172.21.15.33:39176) 2024-04-25T01:09:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4cb20640 1 -- 172.21.15.33:0/475312602 learned_addr learned my addr 172.21.15.33:0/475312602 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:30.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 0x7faf480cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2113525250 0 0) 0x7faf480ce280 con 0x7faf480ced40 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faf34003530 con 0x7faf480ced40 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1833072811 0 0) 0x7faf34003530 con 0x7faf480ced40 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 msgr2=0x7faf480cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 0x7faf480cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 >> v1:172.21.15.138:6789/0 conn(0x7faf4815c020 legacy=0x7faf480ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf480d0a40 con 0x7faf480ced40 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7faf380016a0 con 0x7faf480ced40 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 >> v1:172.21.15.33:6789/0 conn(0x7faf480ced40 legacy=0x7faf480cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf340039f0 0x7faf34003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 0x7faf34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf3400bb60 con 0x7faf34004d50 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf3400bce0 con 0x7faf340039f0 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faf3400bee0 con 0x7faf480aff10 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4d321640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf340039f0 0x7faf34003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4cb20640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 0x7faf34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 -- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 msgr2=0x7faf34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf480aff10 0x7faf34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 -- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf340039f0 msgr2=0x7faf34003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf340039f0 0x7faf34003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 -- 172.21.15.33:0/475312602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf34003530 con 0x7faf34004d50 2024-04-25T01:09:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4d321640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf340039f0 0x7faf34003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf4db22640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf340051d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faf40007360 tx=0x7faf4001ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ad0a4be44cc87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf400b2070 con 0x7faf34004d50 2024-04-25T01:09:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf400a5d40 con 0x7faf34004d50 2024-04-25T01:09:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.184+0000 7faf3ffff640 1 -- 172.21.15.33:0/475312602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40009be0 con 0x7faf34004d50 2024-04-25T01:09:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- 172.21.15.33:0/475312602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 msgr2=0x7faf340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 --2- 172.21.15.33:0/475312602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf340051d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faf40007360 tx=0x7faf4001ce40 comp rx=0 tx=0).stop 2024-04-25T01:09:30.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4db22640 1 -- 172.21.15.33:0/475312602 reap_dead start 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- 172.21.15.33:0/475312602 shutdown_connections 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- 172.21.15.33:0/475312602 >> 172.21.15.33:0/475312602 conn(0x7faf480b3770 msgr2=0x7faf480b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- 172.21.15.33:0/475312602 shutdown_connections 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- 172.21.15.33:0/475312602 wait complete. 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 Processor -- start 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- start start 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf48158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf481f1720 0x7faf480b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf480b2190 0x7faf4815ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faf480ce030 con 0x7faf34004d50 2024-04-25T01:09:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faf480b0f30 con 0x7faf480b2190 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7faf480aea70 con 0x7faf481f1720 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf48158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4db22640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf480b2190 0x7faf4815ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf48158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34688/0 (socket says 172.21.15.33:34688) 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 -- 172.21.15.33:0/47167930 learned_addr learned my addr 172.21.15.33:0/47167930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4cb20640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf481f1720 0x7faf480b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 -- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf481f1720 msgr2=0x7faf480b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf481f1720 0x7faf480b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 -- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf480b2190 msgr2=0x7faf4815ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf480b2190 0x7faf4815ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 -- 172.21.15.33:0/47167930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf480d2b00 con 0x7faf34004d50 2024-04-25T01:09:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4d321640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf48158cd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e550 tx=0x7faf4400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf3dffb640 1 -- 172.21.15.33:0/47167930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf440052e0 con 0x7faf34004d50 2024-04-25T01:09:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf3dffb640 1 -- 172.21.15.33:0/47167930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faf44018070 con 0x7faf34004d50 2024-04-25T01:09:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf480c9fe0 con 0x7faf34004d50 2024-04-25T01:09:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf3dffb640 1 -- 172.21.15.33:0/47167930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf44005480 con 0x7faf34004d50 2024-04-25T01:09:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.188+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf480ca560 con 0x7faf34004d50 2024-04-25T01:09:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.192+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf48067b50 con 0x7faf34004d50 2024-04-25T01:09:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.192+0000 7faf3dffb640 1 -- 172.21.15.33:0/47167930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faf44003840 con 0x7faf34004d50 2024-04-25T01:09:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.192+0000 7faf3dffb640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf2c0678f0 0x7faf2c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.192+0000 7faf3dffb640 1 -- 172.21.15.33:0/47167930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7faf44086720 con 0x7faf34004d50 2024-04-25T01:09:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.192+0000 7faf4cb20640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf2c0678f0 0x7faf2c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.192+0000 7faf4cb20640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf2c0678f0 0x7faf2c069db0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf480d1b40 tx=0x7faf38007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.196+0000 7faf3dffb640 1 -- 172.21.15.33:0/47167930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7faf44056860 con 0x7faf34004d50 2024-04-25T01:09:30.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.356+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faf4815d820 con 0x7faf34004d50 2024-04-25T01:09:30.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf3dffb640 1 -- 172.21.15.33:0/47167930 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7faf4405ab60 con 0x7faf34004d50 2024-04-25T01:09:30.351 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:30.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf2c0678f0 msgr2=0x7faf2c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf2c0678f0 0x7faf2c069db0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7faf480d1b40 tx=0x7faf38007040 comp rx=0 tx=0).stop 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 msgr2=0x7faf48158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf48158cd0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faf4400e550 tx=0x7faf4400ea10 comp rx=0 tx=0).stop 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 shutdown_connections 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7faf2c0678f0 0x7faf2c069db0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf480b2190 0x7faf4815ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7faf481f1720 0x7faf480b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 --2- 172.21.15.33:0/47167930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf34004d50 0x7faf48158cd0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 >> 172.21.15.33:0/47167930 conn(0x7faf480b3770 msgr2=0x7faf480d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 shutdown_connections 2024-04-25T01:09:30.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.360+0000 7faf4f585640 1 -- 172.21.15.33:0/47167930 wait complete. 2024-04-25T01:09:30.370 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T01:09:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73aec29640 1 Processor -- start 2024-04-25T01:09:30.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73aec29640 1 -- start start 2024-04-25T01:09:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73aec29640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a80d0310 0x7f73a80b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73aec29640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73a80d26b0 0x7f73a80b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73aec29640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73a80b0750 con 0x7f73a815f6d0 2024-04-25T01:09:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73aec29640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f73a80b0c30 con 0x7f73a80d26b0 2024-04-25T01:09:30.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73aec29640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f73a80b26e0 con 0x7f73a80d0310 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73ad1c6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73a80d26b0 0x7f73a80b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73ad1c6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f73a80d26b0 0x7f73a80b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34692/0 (socket says 172.21.15.33:34692) 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73ad1c6640 1 -- 172.21.15.33:0/2183829306 learned_addr learned my addr 172.21.15.33:0/2183829306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73a7fff640 1 --2- 172.21.15.33:0/2183829306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a80d0310 0x7f73a80b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73ad1c6640 1 -- 172.21.15.33:0/2183829306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a80d0310 msgr2=0x7f73a80b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73ad1c6640 1 --2- 172.21.15.33:0/2183829306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a80d0310 0x7f73a80b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.544+0000 7f73ad1c6640 1 -- 172.21.15.33:0/2183829306 >> v1:172.21.15.138:6789/0 conn(0x7f73a815f6d0 legacy=0x7f73a80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73ad1c6640 1 -- 172.21.15.33:0/2183829306 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a80b29c0 con 0x7f73a80d26b0 2024-04-25T01:09:30.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73a7fff640 1 --2- 172.21.15.33:0/2183829306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a80d0310 0x7f73a80b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73ad1c6640 1 --2- 172.21.15.33:0/2183829306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d26b0 0x7f73a80b18e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009e50 tx=0x7f73a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b89a20ca3bd6d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73a77fe640 1 -- 172.21.15.33:0/2183829306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a00a5670 con 0x7f73a80d26b0 2024-04-25T01:09:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73a77fe640 1 -- 172.21.15.33:0/2183829306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73a00a5810 con 0x7f73a80d26b0 2024-04-25T01:09:30.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73a77fe640 1 -- 172.21.15.33:0/2183829306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73a00a5be0 con 0x7f73a80d26b0 2024-04-25T01:09:30.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 -- 172.21.15.33:0/2183829306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d26b0 msgr2=0x7f73a80b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 --2- 172.21.15.33:0/2183829306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d26b0 0x7f73a80b18e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009e50 tx=0x7f73a001c9f0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 -- 172.21.15.33:0/2183829306 shutdown_connections 2024-04-25T01:09:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 --2- 172.21.15.33:0/2183829306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d26b0 0x7f73a80b18e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 --2- 172.21.15.33:0/2183829306 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a80d0310 0x7f73a80b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 -- 172.21.15.33:0/2183829306 >> 172.21.15.33:0/2183829306 conn(0x7f73a80d1b90 msgr2=0x7f73a80d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 -- 172.21.15.33:0/2183829306 shutdown_connections 2024-04-25T01:09:30.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.548+0000 7f73aec29640 1 -- 172.21.15.33:0/2183829306 wait complete. 2024-04-25T01:09:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 Processor -- start 2024-04-25T01:09:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 -- start start 2024-04-25T01:09:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d0310 0x7f73a814bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a80d26b0 0x7f73a814aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a815f6d0 0x7f73a815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8169060 con 0x7f73a80d0310 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f73a8168ee0 con 0x7f73a80d26b0 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f73a81691e0 con 0x7f73a815f6d0 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d0310 0x7f73a814bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d0310 0x7f73a814bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34696/0 (socket says 172.21.15.33:34696) 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 -- 172.21.15.33:0/96649612 learned_addr learned my addr 172.21.15.33:0/96649612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73a7fff640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a80d26b0 0x7f73a814aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ad1c6640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a815f6d0 0x7f73a815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 -- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a815f6d0 msgr2=0x7f73a815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a815f6d0 0x7f73a815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 -- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a80d26b0 msgr2=0x7f73a814aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a80d26b0 0x7f73a814aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 -- 172.21.15.33:0/96649612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73a81598f0 con 0x7f73a80d0310 2024-04-25T01:09:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73ac9c5640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d0310 0x7f73a814bf50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f739400b410 tx=0x7f739400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73a57fa640 1 -- 172.21.15.33:0/96649612 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394015030 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73a57fa640 1 -- 172.21.15.33:0/96649612 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7394004020 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73a814b0f0 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73a57fa640 1 -- 172.21.15.33:0/96649612 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394004450 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73a814b6b0 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73a8069040 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73a57fa640 1 -- 172.21.15.33:0/96649612 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7394018070 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.552+0000 7f73a57fa640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f738c0678c0 0x7f738c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.556+0000 7f73a57fa640 1 -- 172.21.15.33:0/96649612 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7394085bb0 con 0x7f73a80d0310 2024-04-25T01:09:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.556+0000 7f73a7fff640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f738c0678c0 0x7f738c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.556+0000 7f73a7fff640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f738c0678c0 0x7f738c069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73a814c9f0 tx=0x7f739c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.560+0000 7f73a57fa640 1 -- 172.21.15.33:0/96649612 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7394056380 con 0x7f73a80d0310 2024-04-25T01:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.732+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f73a80cf3f0 con 0x7f73a80d0310 2024-04-25T01:09:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.732+0000 7f73a57fa640 1 -- 172.21.15.33:0/96649612 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f7394055c80 con 0x7f73a80d0310 2024-04-25T01:09:30.727 INFO:teuthology.orchestra.run.smithi033.stderr:listed 0 entries 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f738c0678c0 msgr2=0x7f738c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f738c0678c0 0x7f738c069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f73a814c9f0 tx=0x7f739c008040 comp rx=0 tx=0).stop 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d0310 msgr2=0x7f73a814bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d0310 0x7f73a814bf50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f739400b410 tx=0x7f739400b8d0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 shutdown_connections 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f738c0678c0 0x7f738c069d80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f73a815f6d0 0x7f73a815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73a80d26b0 0x7f73a814aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 --2- 172.21.15.33:0/96649612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73a80d0310 0x7f73a814bf50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 >> 172.21.15.33:0/96649612 conn(0x7f73a80d1b90 msgr2=0x7f73a815ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 shutdown_connections 2024-04-25T01:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.736+0000 7f73aec29640 1 -- 172.21.15.33:0/96649612 wait complete. 2024-04-25T01:09:30.746 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-04-25T01:09:30.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:30.746 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T01:09:30.761 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:30.762 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:09:30.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:30.762 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T01:09:30.811 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T01:09:30.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:09:30.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3d1b0640 1 Processor -- start 2024-04-25T01:09:30.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3d1b0640 1 -- start start 2024-04-25T01:09:30.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3d1b0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba3815e470 0x7fba380ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3d1b0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 0x7fba380cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3d1b0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba380d10b0 con 0x7fba380d03e0 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3d1b0640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fba380d1de0 con 0x7fba3815e470 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3d1b0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba380d1390 con 0x7fba380d48d0 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3659c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba3815e470 0x7fba380ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba36d9d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fba380d03e0 0x7fba380d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39184/0 (socket says 172.21.15.33:39184) 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3659c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba3815e470 0x7fba380ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34708/0 (socket says 172.21.15.33:34708) 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba36d9d640 1 -- 172.21.15.33:0/630785578 learned_addr learned my addr 172.21.15.33:0/630785578 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3759e640 1 --2- 172.21.15.33:0/630785578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 0x7fba380cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3659c640 1 -- 172.21.15.33:0/630785578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 msgr2=0x7fba380cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3659c640 1 --2- 172.21.15.33:0/630785578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 0x7fba380cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3659c640 1 -- 172.21.15.33:0/630785578 >> v1:172.21.15.33:6789/0 conn(0x7fba380d03e0 legacy=0x7fba380d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3659c640 1 -- 172.21.15.33:0/630785578 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba380d1670 con 0x7fba3815e470 2024-04-25T01:09:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba3659c640 1 --2- 172.21.15.33:0/630785578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba3815e470 0x7fba380ce9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097c0 tx=0x7fba2c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac36190b7676dda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba35d9b640 1 -- 172.21.15.33:0/630785578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0b0070 con 0x7fba3815e470 2024-04-25T01:09:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba35d9b640 1 -- 172.21.15.33:0/630785578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba2c0ab600 con 0x7fba3815e470 2024-04-25T01:09:30.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.948+0000 7fba35d9b640 1 -- 172.21.15.33:0/630785578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0ab9d0 con 0x7fba3815e470 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- 172.21.15.33:0/630785578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba3815e470 msgr2=0x7fba380ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/630785578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba3815e470 0x7fba380ce9c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0097c0 tx=0x7fba2c01c410 comp rx=0 tx=0).stop 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- 172.21.15.33:0/630785578 shutdown_connections 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/630785578 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 0x7fba380cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/630785578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba3815e470 0x7fba380ce9c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- 172.21.15.33:0/630785578 >> 172.21.15.33:0/630785578 conn(0x7fba380d3db0 msgr2=0x7fba380d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- 172.21.15.33:0/630785578 shutdown_connections 2024-04-25T01:09:30.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- 172.21.15.33:0/630785578 wait complete. 2024-04-25T01:09:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 Processor -- start 2024-04-25T01:09:30.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- start start 2024-04-25T01:09:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba380d03e0 0x7fba380b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 0x7fba38160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba3815e470 0x7fba38156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba3816adc0 con 0x7fba380d03e0 2024-04-25T01:09:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba380cfe60 con 0x7fba3815e470 2024-04-25T01:09:30.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba3816af40 con 0x7fba380d48d0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba380d03e0 0x7fba380b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba380d03e0 0x7fba380b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34722/0 (socket says 172.21.15.33:34722) 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 -- 172.21.15.33:0/3141394951 learned_addr learned my addr 172.21.15.33:0/3141394951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 -- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 msgr2=0x7fba38160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 0x7fba38160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 -- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba3815e470 msgr2=0x7fba38156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba3815e470 0x7fba38156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 -- 172.21.15.33:0/3141394951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba380c9f40 con 0x7fba380d03e0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba36d9d640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba380d03e0 0x7fba380b6980 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c450 tx=0x7fba2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba1f7fe640 1 -- 172.21.15.33:0/3141394951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28017070 con 0x7fba380d03e0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba1f7fe640 1 -- 172.21.15.33:0/3141394951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba28005cf0 con 0x7fba380d03e0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba380ca220 con 0x7fba380d03e0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.952+0000 7fba1f7fe640 1 -- 172.21.15.33:0/3141394951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba28007290 con 0x7fba380d03e0 2024-04-25T01:09:30.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.956+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba38161300 con 0x7fba380d03e0 2024-04-25T01:09:30.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.956+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba38155ed0 con 0x7fba380d03e0 2024-04-25T01:09:30.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.956+0000 7fba1f7fe640 1 -- 172.21.15.33:0/3141394951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba2801a070 con 0x7fba380d03e0 2024-04-25T01:09:30.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.956+0000 7fba1f7fe640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba240678c0 0x7fba24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.956+0000 7fba1f7fe640 1 -- 172.21.15.33:0/3141394951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fba280853b0 con 0x7fba380d03e0 2024-04-25T01:09:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.956+0000 7fba3659c640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba240678c0 0x7fba24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:30.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.956+0000 7fba3659c640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba240678c0 0x7fba24069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0ad000 tx=0x7fba2c004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:30.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:30.960+0000 7fba1f7fe640 1 -- 172.21.15.33:0/3141394951 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba28055af0 con 0x7fba380d03e0 2024-04-25T01:09:31.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.148+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba38069040 con 0x7fba380d03e0 2024-04-25T01:09:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.148+0000 7fba1f7fe640 1 -- 172.21.15.33:0/3141394951 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fba280553f0 con 0x7fba380d03e0 2024-04-25T01:09:31.142 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:31.142 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:09:31.142 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:09:31.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba240678c0 msgr2=0x7fba24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba240678c0 0x7fba24069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fba2c0ad000 tx=0x7fba2c004740 comp rx=0 tx=0).stop 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba380d03e0 msgr2=0x7fba380b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba380d03e0 0x7fba380b6980 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fba2800c450 tx=0x7fba2800c910 comp rx=0 tx=0).stop 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 shutdown_connections 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba240678c0 0x7fba24069d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba3815e470 0x7fba38156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba380d48d0 0x7fba38160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 --2- 172.21.15.33:0/3141394951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba380d03e0 0x7fba380b6980 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 >> 172.21.15.33:0/3141394951 conn(0x7fba380d3db0 msgr2=0x7fba3815b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 shutdown_connections 2024-04-25T01:09:31.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.152+0000 7fba3d1b0640 1 -- 172.21.15.33:0/3141394951 wait complete. 2024-04-25T01:09:31.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff20389a640 1 Processor -- start 2024-04-25T01:09:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff20389a640 1 -- start start 2024-04-25T01:09:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff20389a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 0x7ff1fc16d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff20389a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1fc0cf480 con 0x7ff1fc165d60 2024-04-25T01:09:31.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff20389a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1fc0cf760 con 0x7ff1fc0d49c0 2024-04-25T01:09:31.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff20389a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff1fc0cfa40 con 0x7ff1fc0c8de0 2024-04-25T01:09:31.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff201636640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff1fc165d60 0x7ff1fc0d0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39194/0 (socket says 172.21.15.33:39194) 2024-04-25T01:09:31.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.288+0000 7ff201636640 1 -- 172.21.15.33:0/1106566997 learned_addr learned my addr 172.21.15.33:0/1106566997 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 0x7ff1fc16d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252082828 0 0) 0x7ff1fc0cf480 con 0x7ff1fc165d60 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1e8003530 con 0x7ff1fc165d60 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3133520085 0 0) 0x7ff1e8003530 con 0x7ff1fc165d60 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 msgr2=0x7ff1fc16d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 0x7ff1fc16d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 >> v1:172.21.15.138:6789/0 conn(0x7ff1fc0d49c0 legacy=0x7ff1fc0d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc0cfd20 con 0x7ff1fc165d60 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7ff1f8001680 con 0x7ff1fc165d60 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 >> v1:172.21.15.33:6789/0 conn(0x7ff1fc165d60 legacy=0x7ff1fc0d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 0x7ff1e80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8003fc0 0x7ff1e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e8004d50 0x7ff1e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc0cfa40 con 0x7ff1e8004d50 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e800bb10 con 0x7ff1e8003fc0 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e800bd10 con 0x7ff1fc0c8de0 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e8004d50 0x7ff1e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201636640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 0x7ff1e80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff200e35640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8003fc0 0x7ff1e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 -- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 msgr2=0x7ff1e80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0c8de0 0x7ff1e80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 -- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8003fc0 msgr2=0x7ff1e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8003fc0 0x7ff1e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 -- 172.21.15.33:0/1106566997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e8003530 con 0x7ff1e8004d50 2024-04-25T01:09:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e8004d50 0x7ff1e80051d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009f90 tx=0x7ff1f401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec5660388470e91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40b2070 con 0x7ff1e8004d50 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f40a5c50 con 0x7ff1e8004d50 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff1f3fff640 1 -- 172.21.15.33:0/1106566997 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f4009c10 con 0x7ff1e8004d50 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff20389a640 1 -- 172.21.15.33:0/1106566997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e8004d50 msgr2=0x7ff1e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff20389a640 1 --2- 172.21.15.33:0/1106566997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1e8004d50 0x7ff1e80051d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009f90 tx=0x7ff1f401cc40 comp rx=0 tx=0).stop 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff201e37640 1 -- 172.21.15.33:0/1106566997 reap_dead start 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff20389a640 1 -- 172.21.15.33:0/1106566997 shutdown_connections 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.292+0000 7ff20389a640 1 -- 172.21.15.33:0/1106566997 >> 172.21.15.33:0/1106566997 conn(0x7ff1fc0d3e30 msgr2=0x7ff1fc0d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- 172.21.15.33:0/1106566997 shutdown_connections 2024-04-25T01:09:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- 172.21.15.33:0/1106566997 wait complete. 2024-04-25T01:09:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 Processor -- start 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- start start 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8004d50 0x7ff1fc162310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1fc0b38f0 0x7ff1fc1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0b5c80 0x7ff1fc0b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc16b1e0 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc0d2cb0 con 0x7ff1e8004d50 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff1fc16b360 con 0x7ff1fc0b5c80 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1fc0b38f0 0x7ff1fc1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1fc0b38f0 0x7ff1fc1501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34736/0 (socket says 172.21.15.33:34736) 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 -- 172.21.15.33:0/1690852871 learned_addr learned my addr 172.21.15.33:0/1690852871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff201e37640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0b5c80 0x7ff1fc0b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff201636640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8004d50 0x7ff1fc162310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 -- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0b5c80 msgr2=0x7ff1fc0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0b5c80 0x7ff1fc0b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 -- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8004d50 msgr2=0x7ff1fc162310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8004d50 0x7ff1fc162310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 -- 172.21.15.33:0/1690852871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc0caa10 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff200e35640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1fc0b38f0 0x7ff1fc1501f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00f620 tx=0x7ff1ec00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff1f1ffb640 1 -- 172.21.15.33:0/1690852871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec01a070 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff1f1ffb640 1 -- 172.21.15.33:0/1690852871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec005320 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff1f1ffb640 1 -- 172.21.15.33:0/1690852871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec007250 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc0b4120 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc0b46e0 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.296+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc069040 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.300+0000 7ff1f1ffb640 1 -- 172.21.15.33:0/1690852871 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff1ec01d070 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.300+0000 7ff1f1ffb640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1e00678c0 0x7ff1e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.300+0000 7ff1f1ffb640 1 -- 172.21.15.33:0/1690852871 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff1ec085790 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.300+0000 7ff201636640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1e00678c0 0x7ff1e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.300+0000 7ff201636640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1e00678c0 0x7ff1e0069d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8001060 tx=0x7ff1f8000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.304+0000 7ff1f1ffb640 1 -- 172.21.15.33:0/1690852871 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff1ec0557e0 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.468+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff1fc0b56b0 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff1f1ffb640 1 -- 172.21.15.33:0/1690852871 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7ff1ec059ae0 con 0x7ff1fc0b38f0 2024-04-25T01:09:31.465 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:31.465 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1e00678c0 msgr2=0x7ff1e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1e00678c0 0x7ff1e0069d80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1f8001060 tx=0x7ff1f8000ff0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1fc0b38f0 msgr2=0x7ff1fc1501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1fc0b38f0 0x7ff1fc1501f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00f620 tx=0x7ff1ec00fae0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 shutdown_connections 2024-04-25T01:09:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff1e00678c0 0x7ff1e0069d80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff1fc0b5c80 0x7ff1fc0b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1fc0b38f0 0x7ff1fc1501f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 --2- 172.21.15.33:0/1690852871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1e8004d50 0x7ff1fc162310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.472+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 >> 172.21.15.33:0/1690852871 conn(0x7ff1fc0d3e30 msgr2=0x7ff1fc0c92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.476+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 shutdown_connections 2024-04-25T01:09:31.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.476+0000 7ff20389a640 1 -- 172.21.15.33:0/1690852871 wait complete. 2024-04-25T01:09:31.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:08:21.975016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:08:22.806669+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T01:08:26.528967+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:09:31.484 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 Processor -- start 2024-04-25T01:09:31.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 -- start start 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0168144610 0x7f01680b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016806dde0 0x7f016806f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f016806f8c0 0x7f01680ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f016806eef0 con 0x7f0168144610 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f01680b5710 con 0x7f016806f8c0 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f016dc86640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f016806e2e0 con 0x7f016806dde0 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0168144610 0x7f01680b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0168144610 0x7f01680b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34738/0 (socket says 172.21.15.33:34738) 2024-04-25T01:09:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 -- 172.21.15.33:0/2014656416 learned_addr learned my addr 172.21.15.33:0/2014656416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f0167fff640 1 --2- 172.21.15.33:0/2014656416 >> v2:172.21.15.138:3300/0 conn(0x7f016806f8c0 0x7f01680ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f0166ffd640 1 --2- 172.21.15.33:0/2014656416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016806dde0 0x7f016806f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 -- 172.21.15.33:0/2014656416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016806dde0 msgr2=0x7f016806f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 --2- 172.21.15.33:0/2014656416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016806dde0 0x7f016806f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 -- 172.21.15.33:0/2014656416 >> v2:172.21.15.138:3300/0 conn(0x7f016806f8c0 msgr2=0x7f01680ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 --2- 172.21.15.33:0/2014656416 >> v2:172.21.15.138:3300/0 conn(0x7f016806f8c0 0x7f01680ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.608+0000 7f01677fe640 1 -- 172.21.15.33:0/2014656416 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016806e5c0 con 0x7f0168144610 2024-04-25T01:09:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f01677fe640 1 --2- 172.21.15.33:0/2014656416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168144610 0x7f01680b51d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f01680b6210 tx=0x7f016001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba5f3de49c46c5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f01667fc640 1 -- 172.21.15.33:0/2014656416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600a5780 con 0x7f0168144610 2024-04-25T01:09:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f01667fc640 1 -- 172.21.15.33:0/2014656416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01600a5920 con 0x7f0168144610 2024-04-25T01:09:31.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f01667fc640 1 -- 172.21.15.33:0/2014656416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600a5cf0 con 0x7f0168144610 2024-04-25T01:09:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 -- 172.21.15.33:0/2014656416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168144610 msgr2=0x7f01680b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 --2- 172.21.15.33:0/2014656416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168144610 0x7f01680b51d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f01680b6210 tx=0x7f016001c9f0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 -- 172.21.15.33:0/2014656416 shutdown_connections 2024-04-25T01:09:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 --2- 172.21.15.33:0/2014656416 >> v2:172.21.15.138:3300/0 conn(0x7f016806f8c0 0x7f01680ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 --2- 172.21.15.33:0/2014656416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f016806dde0 0x7f016806f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 --2- 172.21.15.33:0/2014656416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0168144610 0x7f01680b51d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 -- 172.21.15.33:0/2014656416 >> 172.21.15.33:0/2014656416 conn(0x7f01680b2820 msgr2=0x7f01680b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 -- 172.21.15.33:0/2014656416 shutdown_connections 2024-04-25T01:09:31.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 -- 172.21.15.33:0/2014656416 wait complete. 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 Processor -- start 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.612+0000 7f016dc86640 1 -- start start 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f016806dde0 0x7f016815dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016806f8c0 0x7f01681632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0168144610 0x7f0168140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f016815e9e0 con 0x7f016806dde0 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f016815e860 con 0x7f016806f8c0 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f016815eb60 con 0x7f0168144610 2024-04-25T01:09:31.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f016806dde0 0x7f016815dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f016806dde0 0x7f016815dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34748/0 (socket says 172.21.15.33:34748) 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 -- 172.21.15.33:0/1745389351 learned_addr learned my addr 172.21.15.33:0/1745389351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f0167fff640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0168144610 0x7f0168140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f0166ffd640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016806f8c0 0x7f01681632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 -- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0168144610 msgr2=0x7f0168140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0168144610 0x7f0168140eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 -- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016806f8c0 msgr2=0x7f01681632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016806f8c0 0x7f01681632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 -- 172.21.15.33:0/1745389351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01680c6a60 con 0x7f016806dde0 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f01677fe640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f016806dde0 0x7f016815dfd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f016001b0a0 tx=0x7f0160019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f015ffff640 1 -- 172.21.15.33:0/1745389351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600b8470 con 0x7f016806dde0 2024-04-25T01:09:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f015ffff640 1 -- 172.21.15.33:0/1745389351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01600b8650 con 0x7f016806dde0 2024-04-25T01:09:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01680c6d40 con 0x7f016806dde0 2024-04-25T01:09:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f015ffff640 1 -- 172.21.15.33:0/1745389351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01600b8a40 con 0x7f016806dde0 2024-04-25T01:09:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01680c7300 con 0x7f016806dde0 2024-04-25T01:09:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0168069040 con 0x7f016806dde0 2024-04-25T01:09:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f015ffff640 1 -- 172.21.15.33:0/1745389351 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01600c0030 con 0x7f016806dde0 2024-04-25T01:09:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f015ffff640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f01480678f0 0x7f0148069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.616+0000 7f015ffff640 1 -- 172.21.15.33:0/1745389351 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f01601207b0 con 0x7f016806dde0 2024-04-25T01:09:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.620+0000 7f0166ffd640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f01480678f0 0x7f0148069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.620+0000 7f0166ffd640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f01480678f0 0x7f0148069db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01680cd380 tx=0x7f0150008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.620+0000 7f015ffff640 1 -- 172.21.15.33:0/1745389351 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f01600c7060 con 0x7f016806dde0 2024-04-25T01:09:31.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0168072a60 con 0x7f016806dde0 2024-04-25T01:09:31.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f015ffff640 1 -- 172.21.15.33:0/1745389351 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f01600165e0 con 0x7f016806dde0 2024-04-25T01:09:31.777 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f01480678f0 msgr2=0x7f0148069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f01480678f0 0x7f0148069db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01680cd380 tx=0x7f0150008040 comp rx=0 tx=0).stop 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f016806dde0 msgr2=0x7f016815dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f016806dde0 0x7f016815dfd0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f016001b0a0 tx=0x7f0160019790 comp rx=0 tx=0).stop 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 shutdown_connections 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f01480678f0 0x7f0148069db0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0168144610 0x7f0168140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f016806f8c0 0x7f01681632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 --2- 172.21.15.33:0/1745389351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f016806dde0 0x7f016815dfd0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 >> 172.21.15.33:0/1745389351 conn(0x7f01680b2820 msgr2=0x7f01680cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 shutdown_connections 2024-04-25T01:09:31.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.784+0000 7f016dc86640 1 -- 172.21.15.33:0/1745389351 wait complete. 2024-04-25T01:09:31.796 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:09:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcfa18640 1 Processor -- start 2024-04-25T01:09:31.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcfa18640 1 -- start start 2024-04-25T01:09:31.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcfa18640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcfc80d2ac0 0x7fcfc8168d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcfa18640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc80b6cc0 con 0x7fcfc815e0d0 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcfa18640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcfc80b6f30 con 0x7fcfc80d1f60 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcfa18640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc80cfb00 con 0x7fcfc80d2ac0 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcdfb5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcfc80d2ac0 0x7fcfc8168d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcdfb5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcfc80d2ac0 0x7fcfc8168d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34758/0 (socket says 172.21.15.33:34758) 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcdfb5640 1 -- 172.21.15.33:0/3480011918 learned_addr learned my addr 172.21.15.33:0/3480011918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcd7b4640 1 --1- 172.21.15.33:0/3480011918 >> v1:172.21.15.33:6789/0 conn(0x7fcfc815e0d0 0x7fcfc80cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39208/0 (socket says 172.21.15.33:39208) 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcdfb5640 1 -- 172.21.15.33:0/3480011918 >> v1:172.21.15.138:6789/0 conn(0x7fcfc80d1f60 legacy=0x7fcfc80d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcdfb5640 1 -- 172.21.15.33:0/3480011918 >> v1:172.21.15.33:6789/0 conn(0x7fcfc815e0d0 legacy=0x7fcfc80cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfbffff640 1 -- 172.21.15.33:0/3480011918 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678819180 0 0) 0x7fcfc80b6cc0 con 0x7fcfc815e0d0 2024-04-25T01:09:31.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.920+0000 7fcfcdfb5640 1 -- 172.21.15.33:0/3480011918 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc80b7210 con 0x7fcfc80d2ac0 2024-04-25T01:09:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcdfb5640 1 --2- 172.21.15.33:0/3480011918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d2ac0 0x7fcfc8168d00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009e50 tx=0x7fcfc001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624206f830a898b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfbffff640 1 -- 172.21.15.33:0/3480011918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00a56e0 con 0x7fcfc80d2ac0 2024-04-25T01:09:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfbffff640 1 -- 172.21.15.33:0/3480011918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc00a5880 con 0x7fcfc80d2ac0 2024-04-25T01:09:31.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfbffff640 1 -- 172.21.15.33:0/3480011918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc00a5c50 con 0x7fcfc80d2ac0 2024-04-25T01:09:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- 172.21.15.33:0/3480011918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d2ac0 msgr2=0x7fcfc8168d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/3480011918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d2ac0 0x7fcfc8168d00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcfc0009e50 tx=0x7fcfc001c9f0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- 172.21.15.33:0/3480011918 shutdown_connections 2024-04-25T01:09:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/3480011918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d2ac0 0x7fcfc8168d00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- 172.21.15.33:0/3480011918 >> 172.21.15.33:0/3480011918 conn(0x7fcfc80d3dd0 msgr2=0x7fcfc80d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- 172.21.15.33:0/3480011918 shutdown_connections 2024-04-25T01:09:31.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- 172.21.15.33:0/3480011918 wait complete. 2024-04-25T01:09:31.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 Processor -- start 2024-04-25T01:09:31.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- start start 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d1f60 0x7fcfc80cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcfc80d2ac0 0x7fcfc80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc815e0d0 0x7fcfc8154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc816bdf0 con 0x7fcfc80d1f60 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc816bc70 con 0x7fcfc815e0d0 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcfa18640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc816bf70 con 0x7fcfc80d2ac0 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d1f60 0x7fcfc80cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d1f60 0x7fcfc80cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34764/0 (socket says 172.21.15.33:34764) 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 -- 172.21.15.33:0/2644168714 learned_addr learned my addr 172.21.15.33:0/2644168714 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcdfb5640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc815e0d0 0x7fcfc8154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfccfb3640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcfc80d2ac0 0x7fcfc80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 -- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcfc80d2ac0 msgr2=0x7fcfc80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcfc80d2ac0 0x7fcfc80caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 -- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc815e0d0 msgr2=0x7fcfc8154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc815e0d0 0x7fcfc8154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfcd7b4640 1 -- 172.21.15.33:0/2644168714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc80cb6f0 con 0x7fcfc80d1f60 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.924+0000 7fcfccfb3640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcfc80d2ac0 0x7fcfc80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfcd7b4640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d1f60 0x7fcfc80cc790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c520 tx=0x7fcfc400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfbdffb640 1 -- 172.21.15.33:0/2644168714 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4018070 con 0x7fcfc80d1f60 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfbdffb640 1 -- 172.21.15.33:0/2644168714 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc40074e0 con 0x7fcfc80d1f60 2024-04-25T01:09:31.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc80cb9d0 con 0x7fcfc80d1f60 2024-04-25T01:09:31.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfbdffb640 1 -- 172.21.15.33:0/2644168714 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40049f0 con 0x7fcfc80d1f60 2024-04-25T01:09:31.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc80cbf90 con 0x7fcfc80d1f60 2024-04-25T01:09:31.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc8069040 con 0x7fcfc80d1f60 2024-04-25T01:09:31.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfbdffb640 1 -- 172.21.15.33:0/2644168714 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcfc401b070 con 0x7fcfc80d1f60 2024-04-25T01:09:31.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfbdffb640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfac0678c0 0x7fcfac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:31.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfbdffb640 1 -- 172.21.15.33:0/2644168714 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcfc4085440 con 0x7fcfc80d1f60 2024-04-25T01:09:31.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfccfb3640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfac0678c0 0x7fcfac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:31.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.928+0000 7fcfccfb3640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfac0678c0 0x7fcfac069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80b66b0 tx=0x7fcfb8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:31.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:31.932+0000 7fcfbdffb640 1 -- 172.21.15.33:0/2644168714 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcfc4055c10 con 0x7fcfc80d1f60 2024-04-25T01:09:32.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.092+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcfc81618d0 con 0x7fcfc80d1f60 2024-04-25T01:09:32.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.092+0000 7fcfbdffb640 1 -- 172.21.15.33:0/2644168714 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fcfc4055510 con 0x7fcfc80d1f60 2024-04-25T01:09:32.086 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfac0678c0 msgr2=0x7fcfac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfac0678c0 0x7fcfac069d80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80b66b0 tx=0x7fcfb8008040 comp rx=0 tx=0).stop 2024-04-25T01:09:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d1f60 msgr2=0x7fcfc80cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d1f60 0x7fcfc80cc790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400c520 tx=0x7fcfc400c9e0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 shutdown_connections 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcfac0678c0 0x7fcfac069d80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfc815e0d0 0x7fcfc8154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcfc80d2ac0 0x7fcfc80caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 --2- 172.21.15.33:0/2644168714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc80d1f60 0x7fcfc80cc790 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 >> 172.21.15.33:0/2644168714 conn(0x7fcfc80d3dd0 msgr2=0x7fcfc80d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 shutdown_connections 2024-04-25T01:09:32.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.096+0000 7fcfcfa18640 1 -- 172.21.15.33:0/2644168714 wait complete. 2024-04-25T01:09:32.105 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:09:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac40db4640 1 Processor -- start 2024-04-25T01:09:32.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac40db4640 1 -- start start 2024-04-25T01:09:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac40db4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c06eb60 0x7fac3c160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac40db4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac3c071170 0x7fac3c071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac40db4640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac3c0cee10 con 0x7fac3c145660 2024-04-25T01:09:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac40db4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fac3c145a40 con 0x7fac3c071170 2024-04-25T01:09:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac40db4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac3c070ce0 con 0x7fac3c06eb60 2024-04-25T01:09:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac3ad9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac3c071170 0x7fac3c071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac3ad9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fac3c071170 0x7fac3c071600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34772/0 (socket says 172.21.15.33:34772) 2024-04-25T01:09:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac3ad9d640 1 -- 172.21.15.33:0/3768978683 learned_addr learned my addr 172.21.15.33:0/3768978683 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac39d9b640 1 --2- 172.21.15.33:0/3768978683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c06eb60 0x7fac3c160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac3ad9d640 1 -- 172.21.15.33:0/3768978683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c06eb60 msgr2=0x7fac3c160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac3ad9d640 1 --2- 172.21.15.33:0/3768978683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c06eb60 0x7fac3c160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac3ad9d640 1 -- 172.21.15.33:0/3768978683 >> v1:172.21.15.138:6789/0 conn(0x7fac3c145660 legacy=0x7fac3c15f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.228+0000 7fac3ad9d640 1 -- 172.21.15.33:0/3768978683 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c0cf0f0 con 0x7fac3c071170 2024-04-25T01:09:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac3ad9d640 1 --2- 172.21.15.33:0/3768978683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c071170 0x7fac3c071600 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fac34009e50 tx=0x7fac3401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1255080c4fb30d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac3959a640 1 -- 172.21.15.33:0/3768978683 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac340a5840 con 0x7fac3c071170 2024-04-25T01:09:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac3959a640 1 -- 172.21.15.33:0/3768978683 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac340a59e0 con 0x7fac3c071170 2024-04-25T01:09:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac3959a640 1 -- 172.21.15.33:0/3768978683 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac340a5db0 con 0x7fac3c071170 2024-04-25T01:09:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- 172.21.15.33:0/3768978683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c071170 msgr2=0x7fac3c071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 --2- 172.21.15.33:0/3768978683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c071170 0x7fac3c071600 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fac34009e50 tx=0x7fac3401c9c0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- 172.21.15.33:0/3768978683 shutdown_connections 2024-04-25T01:09:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 --2- 172.21.15.33:0/3768978683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c071170 0x7fac3c071600 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 --2- 172.21.15.33:0/3768978683 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c06eb60 0x7fac3c160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- 172.21.15.33:0/3768978683 >> 172.21.15.33:0/3768978683 conn(0x7fac3c0b1810 msgr2=0x7fac3c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- 172.21.15.33:0/3768978683 shutdown_connections 2024-04-25T01:09:32.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- 172.21.15.33:0/3768978683 wait complete. 2024-04-25T01:09:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 Processor -- start 2024-04-25T01:09:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- start start 2024-04-25T01:09:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c06eb60 0x7fac3c140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c071170 0x7fac3c141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c145660 0x7fac3c1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c160730 con 0x7fac3c06eb60 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fac3c1605b0 con 0x7fac3c145660 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.232+0000 7fac40db4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fac3c168ca0 con 0x7fac3c071170 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c06eb60 0x7fac3c140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c06eb60 0x7fac3c140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34788/0 (socket says 172.21.15.33:34788) 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 -- 172.21.15.33:0/1247247918 learned_addr learned my addr 172.21.15.33:0/1247247918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3ad9d640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c145660 0x7fac3c1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 -- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c071170 msgr2=0x7fac3c141530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c071170 0x7fac3c141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 -- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c145660 msgr2=0x7fac3c1356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c145660 0x7fac3c1356b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 -- 172.21.15.33:0/1247247918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3c141b60 con 0x7fac3c06eb60 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3ad9d640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c145660 0x7fac3c1356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac3a59c640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c06eb60 0x7fac3c140ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00c450 tx=0x7fac2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac2affd640 1 -- 172.21.15.33:0/1247247918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c012830 con 0x7fac3c06eb60 2024-04-25T01:09:32.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac2affd640 1 -- 172.21.15.33:0/1247247918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac2c017070 con 0x7fac3c06eb60 2024-04-25T01:09:32.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac3c141e40 con 0x7fac3c06eb60 2024-04-25T01:09:32.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac2affd640 1 -- 172.21.15.33:0/1247247918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c012c30 con 0x7fac3c06eb60 2024-04-25T01:09:32.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac3c0b3190 con 0x7fac3c06eb60 2024-04-25T01:09:32.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac3c069040 con 0x7fac3c06eb60 2024-04-25T01:09:32.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac2affd640 1 -- 172.21.15.33:0/1247247918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac2c01c460 con 0x7fac3c06eb60 2024-04-25T01:09:32.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac2affd640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac1c0678f0 0x7fac1c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac2affd640 1 -- 172.21.15.33:0/1247247918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fac2c085e60 con 0x7fac3c06eb60 2024-04-25T01:09:32.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac39d9b640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac1c0678f0 0x7fac1c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.236+0000 7fac39d9b640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac1c0678f0 0x7fac1c069db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac3c15f320 tx=0x7fac30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.240+0000 7fac2affd640 1 -- 172.21.15.33:0/1247247918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fac2c0565a0 con 0x7fac3c06eb60 2024-04-25T01:09:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.396+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fac3c072ac0 con 0x7fac3c06eb60 2024-04-25T01:09:32.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.396+0000 7fac2affd640 1 -- 172.21.15.33:0/1247247918 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fac2c055ea0 con 0x7fac3c06eb60 2024-04-25T01:09:32.390 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:09:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac1c0678f0 msgr2=0x7fac1c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac1c0678f0 0x7fac1c069db0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fac3c15f320 tx=0x7fac30008040 comp rx=0 tx=0).stop 2024-04-25T01:09:32.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c06eb60 msgr2=0x7fac3c140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c06eb60 0x7fac3c140ff0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00c450 tx=0x7fac2c00c910 comp rx=0 tx=0).stop 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 shutdown_connections 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fac1c0678f0 0x7fac1c069db0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac3c145660 0x7fac3c1356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fac3c071170 0x7fac3c141530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 --2- 172.21.15.33:0/1247247918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac3c06eb60 0x7fac3c140ff0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 >> 172.21.15.33:0/1247247918 conn(0x7fac3c0b1810 msgr2=0x7fac3c13b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 shutdown_connections 2024-04-25T01:09:32.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.400+0000 7fac40db4640 1 -- 172.21.15.33:0/1247247918 wait complete. 2024-04-25T01:09:32.408 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T01:09:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150f669640 1 Processor -- start 2024-04-25T01:09:32.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150f669640 1 -- start start 2024-04-25T01:09:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150f669640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f15080b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150f669640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1508067030 0x7f150806b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150f669640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15080b1410 con 0x7f1508072910 2024-04-25T01:09:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150f669640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f15080b61b0 con 0x7f1508067030 2024-04-25T01:09:32.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150f669640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f150806bb30 con 0x7f150806b210 2024-04-25T01:09:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150d405640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f15080b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150cc04640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1508067030 0x7f150806b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150d405640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f15080b4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:55296/0 (socket says 172.21.15.33:55296) 2024-04-25T01:09:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150d405640 1 -- 172.21.15.33:0/283785491 learned_addr learned my addr 172.21.15.33:0/283785491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150cc04640 1 -- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 msgr2=0x7f15080b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:32.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150cc04640 1 --2- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f15080b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150cc04640 1 -- 172.21.15.33:0/283785491 >> v1:172.21.15.138:6789/0 conn(0x7f1508072910 legacy=0x7f1508072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150cc04640 1 -- 172.21.15.33:0/283785491 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15080b55a0 con 0x7f1508067030 2024-04-25T01:09:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.528+0000 7f150d405640 1 --2- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f15080b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:09:32.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150cc04640 1 --2- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 0x7f150806b5f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097c0 tx=0x7f14f801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89a977b98342e0df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f14fffff640 1 -- 172.21.15.33:0/283785491 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80b1070 con 0x7f1508067030 2024-04-25T01:09:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f14fffff640 1 -- 172.21.15.33:0/283785491 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f80a8050 con 0x7f1508067030 2024-04-25T01:09:32.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f14fffff640 1 -- 172.21.15.33:0/283785491 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80ac860 con 0x7f1508067030 2024-04-25T01:09:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 msgr2=0x7f150806b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 --2- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 0x7f150806b5f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f14f80097c0 tx=0x7f14f801c3e0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- 172.21.15.33:0/283785491 shutdown_connections 2024-04-25T01:09:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 --2- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 0x7f150806b5f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 --2- 172.21.15.33:0/283785491 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f15080b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- 172.21.15.33:0/283785491 >> 172.21.15.33:0/283785491 conn(0x7f15080b1e60 msgr2=0x7f15080b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- 172.21.15.33:0/283785491 shutdown_connections 2024-04-25T01:09:32.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- 172.21.15.33:0/283785491 wait complete. 2024-04-25T01:09:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 Processor -- start 2024-04-25T01:09:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- start start 2024-04-25T01:09:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 0x7f15080999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f1508166ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1508072910 0x7f15080c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1508067490 con 0x7f1508072910 2024-04-25T01:09:32.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15080ca8e0 con 0x7f1508067030 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150f669640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15080cc590 con 0x7f150806b210 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150dc06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1508072910 0x7f15080c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150dc06640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1508072910 0x7f15080c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34796/0 (socket says 172.21.15.33:34796) 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.532+0000 7f150dc06640 1 -- 172.21.15.33:0/1252636004 learned_addr learned my addr 172.21.15.33:0/1252636004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150d405640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 0x7f15080999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150cc04640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f1508166ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150dc06640 1 -- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 msgr2=0x7f1508166ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150dc06640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f1508166ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150dc06640 1 -- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 msgr2=0x7f15080999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150dc06640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 0x7f15080999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150dc06640 1 -- 172.21.15.33:0/1252636004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15081677c0 con 0x7f1508072910 2024-04-25T01:09:32.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150dc06640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1508072910 0x7f15080c8680 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f150400c460 tx=0x7f150400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f14fdffb640 1 -- 172.21.15.33:0/1252636004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504016020 con 0x7f1508072910 2024-04-25T01:09:32.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f14fdffb640 1 -- 172.21.15.33:0/1252636004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1504005220 con 0x7f1508072910 2024-04-25T01:09:32.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1508167aa0 con 0x7f1508072910 2024-04-25T01:09:32.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f14fdffb640 1 -- 172.21.15.33:0/1252636004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1504007030 con 0x7f1508072910 2024-04-25T01:09:32.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1508162cf0 con 0x7f1508072910 2024-04-25T01:09:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1508069040 con 0x7f1508072910 2024-04-25T01:09:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f14fdffb640 1 -- 172.21.15.33:0/1252636004 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f15040071d0 con 0x7f1508072910 2024-04-25T01:09:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f14fdffb640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f14ec0678c0 0x7f14ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f14fdffb640 1 -- 172.21.15.33:0/1252636004 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1504085820 con 0x7f1508072910 2024-04-25T01:09:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.536+0000 7f150d405640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f14ec0678c0 0x7f14ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.540+0000 7f150d405640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f14ec0678c0 0x7f14ec069d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15081688f0 tx=0x7f150000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.540+0000 7f14fdffb640 1 -- 172.21.15.33:0/1252636004 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1504020020 con 0x7f1508072910 2024-04-25T01:09:32.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.696+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1508072d70 con 0x7f1508072910 2024-04-25T01:09:32.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.696+0000 7f14fdffb640 1 -- 172.21.15.33:0/1252636004 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f15040074c0 con 0x7f1508072910 2024-04-25T01:09:32.690 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f14ec0678c0 msgr2=0x7f14ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f14ec0678c0 0x7f14ec069d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f15081688f0 tx=0x7f150000b040 comp rx=0 tx=0).stop 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1508072910 msgr2=0x7f15080c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1508072910 0x7f15080c8680 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f150400c460 tx=0x7f150400c920 comp rx=0 tx=0).stop 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 shutdown_connections 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f14ec0678c0 0x7f14ec069d80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1508072910 0x7f15080c8680 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f150806b210 0x7f1508166ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 --2- 172.21.15.33:0/1252636004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1508067030 0x7f15080999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 >> 172.21.15.33:0/1252636004 conn(0x7f15080b1e60 msgr2=0x7f15080ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 shutdown_connections 2024-04-25T01:09:32.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.700+0000 7f150f669640 1 -- 172.21.15.33:0/1252636004 wait complete. 2024-04-25T01:09:32.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:09:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551902e640 1 Processor -- start 2024-04-25T01:09:32.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551902e640 1 -- start start 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551902e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5514145470 0x7f551415c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551902e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf0e0 0x7f55140b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551902e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551415f9a0 con 0x7f55140cf540 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551902e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f551415f750 con 0x7f5514145470 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551902e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f551415fc80 con 0x7f55140cf0e0 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f5512d9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5514145470 0x7f551415c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f5512d9d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f5514145470 0x7f551415c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34808/0 (socket says 172.21.15.33:34808) 2024-04-25T01:09:32.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f5512d9d640 1 -- 172.21.15.33:0/853619393 learned_addr learned my addr 172.21.15.33:0/853619393 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f551259c640 1 --2- 172.21.15.33:0/853619393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf0e0 0x7f55140b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f5512d9d640 1 -- 172.21.15.33:0/853619393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf0e0 msgr2=0x7f55140b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f5512d9d640 1 --2- 172.21.15.33:0/853619393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf0e0 0x7f55140b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f5512d9d640 1 -- 172.21.15.33:0/853619393 >> v1:172.21.15.138:6789/0 conn(0x7f55140cf540 legacy=0x7f5514071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.828+0000 7f5512d9d640 1 -- 172.21.15.33:0/853619393 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551415ff60 con 0x7f5514145470 2024-04-25T01:09:32.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.832+0000 7f5512d9d640 1 --2- 172.21.15.33:0/853619393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f551415c160 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5504009870 tx=0x7f550401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89344809e9fc492c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f5511d9b640 1 -- 172.21.15.33:0/853619393 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040a7070 con 0x7f5514145470 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f5511d9b640 1 -- 172.21.15.33:0/853619393 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55040b0070 con 0x7f5514145470 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f5511d9b640 1 -- 172.21.15.33:0/853619393 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55040ab890 con 0x7f5514145470 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- 172.21.15.33:0/853619393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 msgr2=0x7f551415c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 --2- 172.21.15.33:0/853619393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f551415c160 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5504009870 tx=0x7f550401c560 comp rx=0 tx=0).stop 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- 172.21.15.33:0/853619393 shutdown_connections 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 --2- 172.21.15.33:0/853619393 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf0e0 0x7f55140b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 --2- 172.21.15.33:0/853619393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f551415c160 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- 172.21.15.33:0/853619393 >> 172.21.15.33:0/853619393 conn(0x7f55140b1810 msgr2=0x7f55140b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- 172.21.15.33:0/853619393 shutdown_connections 2024-04-25T01:09:32.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- 172.21.15.33:0/853619393 wait complete. 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 Processor -- start 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- start start 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55140cf0e0 0x7f55140a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf540 0x7f551413d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f5514143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5514168c70 con 0x7f5514145470 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5514168af0 con 0x7f55140cf0e0 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f5514168df0 con 0x7f55140cf540 2024-04-25T01:09:32.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f5514143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f5514143670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34812/0 (socket says 172.21.15.33:34812) 2024-04-25T01:09:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 -- 172.21.15.33:0/4082837114 learned_addr learned my addr 172.21.15.33:0/4082837114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551259c640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf540 0x7f551413d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f5512d9d640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55140cf0e0 0x7f55140a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 -- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf540 msgr2=0x7f551413d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf540 0x7f551413d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 -- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55140cf0e0 msgr2=0x7f55140a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55140cf0e0 0x7f55140a9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 -- 172.21.15.33:0/4082837114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5514142e90 con 0x7f5514145470 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551359e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f5514143670 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f550c00c730 tx=0x7f550c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f54fb7fe640 1 -- 172.21.15.33:0/4082837114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c003b80 con 0x7f5514145470 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f54fb7fe640 1 -- 172.21.15.33:0/4082837114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550c003d20 con 0x7f5514145470 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551413e350 con 0x7f5514145470 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f54fb7fe640 1 -- 172.21.15.33:0/4082837114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c00aa30 con 0x7f5514145470 2024-04-25T01:09:32.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.836+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5514143150 con 0x7f5514145470 2024-04-25T01:09:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.840+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5514143350 con 0x7f5514145470 2024-04-25T01:09:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.840+0000 7f54fb7fe640 1 -- 172.21.15.33:0/4082837114 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f550c00abd0 con 0x7f5514145470 2024-04-25T01:09:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.840+0000 7f54fb7fe640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f55000678c0 0x7f5500069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:32.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.840+0000 7f54fb7fe640 1 -- 172.21.15.33:0/4082837114 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f550c085eb0 con 0x7f5514145470 2024-04-25T01:09:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.840+0000 7f5512d9d640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f55000678c0 0x7f5500069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:32.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.840+0000 7f5512d9d640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f55000678c0 0x7f5500069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55040af000 tx=0x7f5504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:32.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:32.844+0000 7f54fb7fe640 1 -- 172.21.15.33:0/4082837114 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f5514143350 con 0x7f5514145470 2024-04-25T01:09:33.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5514069040 con 0x7f5514145470 2024-04-25T01:09:33.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f54fb7fe640 1 -- 172.21.15.33:0/4082837114 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7f550c056680 con 0x7f5514145470 2024-04-25T01:09:33.028 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:33.028 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:09:33.028 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f55000678c0 msgr2=0x7f5500069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f55000678c0 0x7f5500069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f55040af000 tx=0x7f5504008040 comp rx=0 tx=0).stop 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 msgr2=0x7f5514143670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f5514143670 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f550c00c730 tx=0x7f550c00cbf0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 shutdown_connections 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f55000678c0 0x7f5500069d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5514145470 0x7f5514143670 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f55140cf540 0x7f551413d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 --2- 172.21.15.33:0/4082837114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55140cf0e0 0x7f55140a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 >> 172.21.15.33:0/4082837114 conn(0x7f55140b1810 msgr2=0x7f55140af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:33.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 shutdown_connections 2024-04-25T01:09:33.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.036+0000 7f551902e640 1 -- 172.21.15.33:0/4082837114 wait complete. 2024-04-25T01:09:33.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe3030bb640 1 Processor -- start 2024-04-25T01:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe3030bb640 1 -- start start 2024-04-25T01:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe3030bb640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2fc0d48d0 0x7fe2fc0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe3030bb640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc0d10b0 con 0x7fe2fc15e470 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe3030bb640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc0d1390 con 0x7fe2fc0d03e0 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe3030bb640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fe2fc0d1de0 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe301658640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe2fc0d48d0 0x7fe2fc0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.168+0000 7fe2fbfff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fe2fc15e470 0x7fe2fc0ce9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39218/0 (socket says 172.21.15.33:39218) 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe2fbfff640 1 -- 172.21.15.33:0/3259171300 learned_addr learned my addr 172.21.15.33:0/3259171300 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe301658640 1 -- 172.21.15.33:0/3259171300 >> v1:172.21.15.138:6789/0 conn(0x7fe2fc0d03e0 legacy=0x7fe2fc0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe301658640 1 -- 172.21.15.33:0/3259171300 >> v1:172.21.15.33:6789/0 conn(0x7fe2fc15e470 legacy=0x7fe2fc0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe301658640 1 -- 172.21.15.33:0/3259171300 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc0d1670 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe301658640 1 --2- 172.21.15.33:0/3259171300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc0cf0d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40097c0 tx=0x7fe2f401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e13721d0526c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe2fb7fe640 1 -- 172.21.15.33:0/3259171300 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40b0070 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe2fb7fe640 1 -- 172.21.15.33:0/3259171300 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f40ab600 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe2fb7fe640 1 -- 172.21.15.33:0/3259171300 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40ab9d0 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe3030bb640 1 -- 172.21.15.33:0/3259171300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 msgr2=0x7fe2fc0cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe3030bb640 1 --2- 172.21.15.33:0/3259171300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc0cf0d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40097c0 tx=0x7fe2f401c3d0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe3030bb640 1 -- 172.21.15.33:0/3259171300 shutdown_connections 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe3030bb640 1 --2- 172.21.15.33:0/3259171300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc0cf0d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe3030bb640 1 -- 172.21.15.33:0/3259171300 >> 172.21.15.33:0/3259171300 conn(0x7fe2fc0d3db0 msgr2=0x7fe2fc0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe3030bb640 1 -- 172.21.15.33:0/3259171300 shutdown_connections 2024-04-25T01:09:33.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.172+0000 7fe3030bb640 1 -- 172.21.15.33:0/3259171300 wait complete. 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 Processor -- start 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 -- start start 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2fc0d03e0 0x7fe2fc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2fc15e470 0x7fe2fc150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc16c160 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc16bfe0 con 0x7fe2fc15e470 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc16c2e0 con 0x7fe2fc0d03e0 2024-04-25T01:09:33.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34826/0 (socket says 172.21.15.33:34826) 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 -- 172.21.15.33:0/2200435301 learned_addr learned my addr 172.21.15.33:0/2200435301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 -- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2fc0d03e0 msgr2=0x7fe2fc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe300e57640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2fc0d03e0 0x7fe2fc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe301658640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2fc15e470 0x7fe2fc150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2fc0d03e0 0x7fe2fc15b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 -- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2fc15e470 msgr2=0x7fe2fc150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2fc15e470 0x7fe2fc150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 -- 172.21.15.33:0/2200435301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc15cf50 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2fbfff640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc150390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000c3a0 tx=0x7fe2f000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2f97fa640 1 -- 172.21.15.33:0/2200435301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0017070 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc15d230 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2f97fa640 1 -- 172.21.15.33:0/2200435301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f0005cf0 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2f97fa640 1 -- 172.21.15.33:0/2200435301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00072f0 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc15b0e0 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2fc069040 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.176+0000 7fe2f97fa640 1 -- 172.21.15.33:0/2200435301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe2f001a070 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.180+0000 7fe2f97fa640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe2e80678c0 0x7fe2e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.180+0000 7fe2f97fa640 1 -- 172.21.15.33:0/2200435301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe2f00853f0 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.180+0000 7fe300e57640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe2e80678c0 0x7fe2e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.180+0000 7fe300e57640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe2e80678c0 0x7fe2e8069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc160a80 tx=0x7fe2ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.184+0000 7fe2f97fa640 1 -- 172.21.15.33:0/2200435301 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe2f0055b20 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.344+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe2fc0d4470 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe2f97fa640 1 -- 172.21.15.33:0/2200435301 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7fe2f0055420 con 0x7fe2fc0d48d0 2024-04-25T01:09:33.340 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:33.340 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:33.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe2e80678c0 msgr2=0x7fe2e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe2e80678c0 0x7fe2e8069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc160a80 tx=0x7fe2ec009040 comp rx=0 tx=0).stop 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 msgr2=0x7fe2fc150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc150390 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000c3a0 tx=0x7fe2f000c860 comp rx=0 tx=0).stop 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 shutdown_connections 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe2e80678c0 0x7fe2e8069d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2fc15e470 0x7fe2fc150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2fc0d48d0 0x7fe2fc150390 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 --2- 172.21.15.33:0/2200435301 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe2fc0d03e0 0x7fe2fc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 >> 172.21.15.33:0/2200435301 conn(0x7fe2fc0d3db0 msgr2=0x7fe2fc156300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 shutdown_connections 2024-04-25T01:09:33.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.348+0000 7fe3030bb640 1 -- 172.21.15.33:0/2200435301 wait complete. 2024-04-25T01:09:33.359 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T01:09:33.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:09:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.476+0000 7fb4fd3d8640 1 Processor -- start 2024-04-25T01:09:33.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.476+0000 7fb4fd3d8640 1 -- start start 2024-04-25T01:09:33.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb4f815dfa0 0x7fb4f80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb4f80d4950 0x7fb4f80d0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4f80d25b0 con 0x7fb4f80d11a0 2024-04-25T01:09:33.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f80d15b0 con 0x7fb4f80d4950 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb4f80d2890 con 0x7fb4f815dfa0 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f77fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb4f80d11a0 0x7fb4f8161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39224/0 (socket says 172.21.15.33:39224) 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f77fe640 1 -- 172.21.15.33:0/1293638692 learned_addr learned my addr 172.21.15.33:0/1293638692 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4ee7fc640 1 --2- 172.21.15.33:0/1293638692 >> v2:172.21.15.33:3300/0 conn(0x7fb4f80d4950 0x7fb4f80d0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f67fc640 1 -- 172.21.15.33:0/1293638692 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2833876814 0 0) 0x7fb4f80d25b0 con 0x7fb4f80d11a0 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f67fc640 1 -- 172.21.15.33:0/1293638692 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4d8003530 con 0x7fb4f80d11a0 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f6ffd640 1 --2- 172.21.15.33:0/1293638692 >> v2:172.21.15.138:3300/0 conn(0x7fb4f815dfa0 0x7fb4f80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4ee7fc640 1 -- 172.21.15.33:0/1293638692 >> v2:172.21.15.138:3300/0 conn(0x7fb4f815dfa0 msgr2=0x7fb4f80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4ee7fc640 1 --2- 172.21.15.33:0/1293638692 >> v2:172.21.15.138:3300/0 conn(0x7fb4f815dfa0 0x7fb4f80d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4ee7fc640 1 -- 172.21.15.33:0/1293638692 >> v1:172.21.15.33:6789/0 conn(0x7fb4f80d11a0 legacy=0x7fb4f8161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4ee7fc640 1 -- 172.21.15.33:0/1293638692 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f80d2b70 con 0x7fb4f80d4950 2024-04-25T01:09:33.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f6ffd640 1 --2- 172.21.15.33:0/1293638692 >> v2:172.21.15.138:3300/0 conn(0x7fb4f815dfa0 0x7fb4f80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4ee7fc640 1 --2- 172.21.15.33:0/1293638692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f80d0c60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8067a60 tx=0x7fb4e801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a5e64174c8c8a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f67fc640 1 -- 172.21.15.33:0/1293638692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80b0070 con 0x7fb4f80d4950 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f67fc640 1 -- 172.21.15.33:0/1293638692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e80ab600 con 0x7fb4f80d4950 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4f67fc640 1 -- 172.21.15.33:0/1293638692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80ab9d0 con 0x7fb4f80d4950 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/1293638692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 msgr2=0x7fb4f80d0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/1293638692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f80d0c60 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8067a60 tx=0x7fb4e801c3f0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/1293638692 shutdown_connections 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/1293638692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f80d0c60 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/1293638692 >> v2:172.21.15.138:3300/0 conn(0x7fb4f815dfa0 0x7fb4f80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/1293638692 >> 172.21.15.33:0/1293638692 conn(0x7fb4f80d3e30 msgr2=0x7fb4f80d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/1293638692 shutdown_connections 2024-04-25T01:09:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.480+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/1293638692 wait complete. 2024-04-25T01:09:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 Processor -- start 2024-04-25T01:09:33.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 -- start start 2024-04-25T01:09:33.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f80d11a0 0x7fb4f815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f8152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4f815dfa0 0x7fb4f8153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f80b0bb0 con 0x7fb4f80d4950 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f80b0a30 con 0x7fb4f80d11a0 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f816aee0 con 0x7fb4f815dfa0 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f8152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f8152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34848/0 (socket says 172.21.15.33:34848) 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 -- 172.21.15.33:0/173511496 learned_addr learned my addr 172.21.15.33:0/173511496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4f6ffd640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f80d11a0 0x7fb4f815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 -- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4f815dfa0 msgr2=0x7fb4f8153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4f77fe640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4f815dfa0 0x7fb4f8153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4f815dfa0 0x7fb4f8153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 -- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f80d11a0 msgr2=0x7fb4f815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f80d11a0 0x7fb4f815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 -- 172.21.15.33:0/173511496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f80b9470 con 0x7fb4f80d4950 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4ee7fc640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f8152f20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80ae040 tx=0x7fb4e8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4effff640 1 -- 172.21.15.33:0/173511496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80b0040 con 0x7fb4f80d4950 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4effff640 1 -- 172.21.15.33:0/173511496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e80b9070 con 0x7fb4f80d4950 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4effff640 1 -- 172.21.15.33:0/173511496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8006e20 con 0x7fb4f80d4950 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4f80cafa0 con 0x7fb4f80d4950 2024-04-25T01:09:33.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.484+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4f80b9750 con 0x7fb4f80d4950 2024-04-25T01:09:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.488+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4f8069040 con 0x7fb4f80d4950 2024-04-25T01:09:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.488+0000 7fb4effff640 1 -- 172.21.15.33:0/173511496 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb4e80b4690 con 0x7fb4f80d4950 2024-04-25T01:09:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.488+0000 7fb4effff640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4dc0678c0 0x7fb4dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.488+0000 7fb4effff640 1 -- 172.21.15.33:0/173511496 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb4e811fba0 con 0x7fb4f80d4950 2024-04-25T01:09:33.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.488+0000 7fb4f6ffd640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4dc0678c0 0x7fb4dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.488+0000 7fb4f6ffd640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4dc0678c0 0x7fb4dc069d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006fd0 tx=0x7fb4e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.492+0000 7fb4effff640 1 -- 172.21.15.33:0/173511496 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb4e8016070 con 0x7fb4f80d4950 2024-04-25T01:09:33.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.680+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4f808dbe0 con 0x7fb4f80d4950 2024-04-25T01:09:33.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4effff640 1 -- 172.21.15.33:0/173511496 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fb4e80f02d0 con 0x7fb4f80d4950 2024-04-25T01:09:33.676 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:33.676 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:09:33.677 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4dc0678c0 msgr2=0x7fb4dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4dc0678c0 0x7fb4dc069d80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006fd0 tx=0x7fb4e0008040 comp rx=0 tx=0).stop 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 msgr2=0x7fb4f8152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f8152f20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb4e80ae040 tx=0x7fb4e8019190 comp rx=0 tx=0).stop 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 shutdown_connections 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb4dc0678c0 0x7fb4dc069d80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb4f815dfa0 0x7fb4f8153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4f80d4950 0x7fb4f8152f20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 --2- 172.21.15.33:0/173511496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb4f80d11a0 0x7fb4f815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 >> 172.21.15.33:0/173511496 conn(0x7fb4f80d3e30 msgr2=0x7fb4f80d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 shutdown_connections 2024-04-25T01:09:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.684+0000 7fb4fd3d8640 1 -- 172.21.15.33:0/173511496 wait complete. 2024-04-25T01:09:33.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:09:33.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81908c640 1 Processor -- start 2024-04-25T01:09:33.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81908c640 1 -- start start 2024-04-25T01:09:33.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81908c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8140b4c50 0x7fe81415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81908c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b5680 0x7fe8140b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81908c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81406f020 con 0x7fe8141563d0 2024-04-25T01:09:33.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81908c640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fe8140b6030 con 0x7fe8140b4c50 2024-04-25T01:09:33.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81908c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe81406f2a0 con 0x7fe8140b5680 2024-04-25T01:09:33.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81259c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8140b4c50 0x7fe81415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81259c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fe8140b4c50 0x7fe81415d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34860/0 (socket says 172.21.15.33:34860) 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81259c640 1 -- 172.21.15.33:0/2186639831 learned_addr learned my addr 172.21.15.33:0/2186639831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81359e640 1 --2- 172.21.15.33:0/2186639831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b5680 0x7fe8140b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81259c640 1 -- 172.21.15.33:0/2186639831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b5680 msgr2=0x7fe8140b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81259c640 1 --2- 172.21.15.33:0/2186639831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b5680 0x7fe8140b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81259c640 1 -- 172.21.15.33:0/2186639831 >> v1:172.21.15.138:6789/0 conn(0x7fe8141563d0 legacy=0x7fe81415cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81259c640 1 -- 172.21.15.33:0/2186639831 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81406f580 con 0x7fe8140b4c50 2024-04-25T01:09:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.816+0000 7fe81359e640 1 --2- 172.21.15.33:0/2186639831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b5680 0x7fe8140b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:33.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81259c640 1 --2- 172.21.15.33:0/2186639831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8140b4c50 0x7fe81415d3f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe808009870 tx=0x7fe80801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ba44b8e304eea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe811d9b640 1 -- 172.21.15.33:0/2186639831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8080b0070 con 0x7fe8140b4c50 2024-04-25T01:09:33.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe811d9b640 1 -- 172.21.15.33:0/2186639831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8080ab600 con 0x7fe8140b4c50 2024-04-25T01:09:33.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe811d9b640 1 -- 172.21.15.33:0/2186639831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8080ab9d0 con 0x7fe8140b4c50 2024-04-25T01:09:33.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- 172.21.15.33:0/2186639831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8140b4c50 msgr2=0x7fe81415d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 --2- 172.21.15.33:0/2186639831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8140b4c50 0x7fe81415d3f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe808009870 tx=0x7fe80801c4d0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- 172.21.15.33:0/2186639831 shutdown_connections 2024-04-25T01:09:33.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 --2- 172.21.15.33:0/2186639831 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b5680 0x7fe8140b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 --2- 172.21.15.33:0/2186639831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8140b4c50 0x7fe81415d3f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- 172.21.15.33:0/2186639831 >> 172.21.15.33:0/2186639831 conn(0x7fe8140734c0 msgr2=0x7fe81409a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:33.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- 172.21.15.33:0/2186639831 shutdown_connections 2024-04-25T01:09:33.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- 172.21.15.33:0/2186639831 wait complete. 2024-04-25T01:09:33.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 Processor -- start 2024-04-25T01:09:33.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- start start 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b4c50 0x7fe814143530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8140b5680 0x7fe8141551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8141563d0 0x7fe814155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe814070f60 con 0x7fe8141563d0 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe814070de0 con 0x7fe8140b5680 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81908c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fe8140710e0 con 0x7fe8140b4c50 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8141563d0 0x7fe814155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8141563d0 0x7fe814155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34876/0 (socket says 172.21.15.33:34876) 2024-04-25T01:09:33.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 -- 172.21.15.33:0/3255665446 learned_addr learned my addr 172.21.15.33:0/3255665446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81259c640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8140b5680 0x7fe8141551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe812d9d640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b4c50 0x7fe814143530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 -- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b4c50 msgr2=0x7fe814143530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b4c50 0x7fe814143530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 -- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8140b5680 msgr2=0x7fe8141551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8140b5680 0x7fe8141551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 -- 172.21.15.33:0/3255665446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe814153820 con 0x7fe8141563d0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe81359e640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8141563d0 0x7fe814155720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe80c007e20 tx=0x7fe80c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe7fb7fe640 1 -- 172.21.15.33:0/3255665446 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c01a070 con 0x7fe8141563d0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.820+0000 7fe7fb7fe640 1 -- 172.21.15.33:0/3255665446 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80c003ba0 con 0x7fe8141563d0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe814153b00 con 0x7fe8141563d0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe7fb7fe640 1 -- 172.21.15.33:0/3255665446 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c00de80 con 0x7fe8141563d0 2024-04-25T01:09:33.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8141540c0 con 0x7fe8141563d0 2024-04-25T01:09:33.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe814069040 con 0x7fe8141563d0 2024-04-25T01:09:33.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe7fb7fe640 1 -- 172.21.15.33:0/3255665446 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe80c01d070 con 0x7fe8141563d0 2024-04-25T01:09:33.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe7fb7fe640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe8000678f0 0x7fe800069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:33.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe7fb7fe640 1 -- 172.21.15.33:0/3255665446 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe80c086240 con 0x7fe8141563d0 2024-04-25T01:09:33.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe812d9d640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe8000678f0 0x7fe800069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:33.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.824+0000 7fe812d9d640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe8000678f0 0x7fe800069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe814148650 tx=0x7fe804009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:33.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:33.828+0000 7fe7fb7fe640 1 -- 172.21.15.33:0/3255665446 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fe80c010070 con 0x7fe8141563d0 2024-04-25T01:09:34.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.024+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8140d2a90 con 0x7fe8141563d0 2024-04-25T01:09:34.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.024+0000 7fe7fb7fe640 1 -- 172.21.15.33:0/3255665446 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5180 (secure 0 0 0) 0x7fe80c056a10 con 0x7fe8141563d0 2024-04-25T01:09:34.018 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:34.018 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4237,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1025247558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1025247558},{"type":"v1","addr":"172.21.15.138:6835","nonce":1025247558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4243,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2470912758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2470912758},{"type":"v1","addr":"172.21.15.138:6837","nonce":2470912758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4245,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6833/4281516030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":4281516030},{"type":"v1","addr":"172.21.15.33:6833","nonce":4281516030}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:08:23.800275+0000","modified":"2024-04-25T01:08:40.521343+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4248},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4248":{"gid":4248,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6835/486086516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":486086516},{"type":"v1","addr":"172.21.15.33:6835","nonce":486086516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4248,"qdb_cluster":[4248]},"id":1}]} 2024-04-25T01:09:34.019 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-04-25T01:09:34.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe8000678f0 msgr2=0x7fe800069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe8000678f0 0x7fe800069db0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe814148650 tx=0x7fe804009040 comp rx=0 tx=0).stop 2024-04-25T01:09:34.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8141563d0 msgr2=0x7fe814155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8141563d0 0x7fe814155720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe80c007e20 tx=0x7fe80c00d4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 shutdown_connections 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fe8000678f0 0x7fe800069db0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe8141563d0 0x7fe814155720 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8140b5680 0x7fe8141551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 --2- 172.21.15.33:0/3255665446 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fe8140b4c50 0x7fe814143530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 >> 172.21.15.33:0/3255665446 conn(0x7fe8140734c0 msgr2=0x7fe814159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 shutdown_connections 2024-04-25T01:09:34.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.028+0000 7fe81908c640 1 -- 172.21.15.33:0/3255665446 wait complete. 2024-04-25T01:09:34.036 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e7d7ba640 1 Processor -- start 2024-04-25T01:09:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e7d7ba640 1 -- start start 2024-04-25T01:09:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e7d7ba640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1e781444f0 0x7f1e780b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e7d7ba640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e781448d0 con 0x7f1e7815e660 2024-04-25T01:09:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e7d7ba640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e780b5b60 con 0x7f1e78072540 2024-04-25T01:09:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e7d7ba640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1e780ce0c0 con 0x7f1e781444f0 2024-04-25T01:09:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e6ffff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1e7815e660 0x7f1e78071e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39226/0 (socket says 172.21.15.33:39226) 2024-04-25T01:09:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e76ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1e781444f0 0x7f1e780b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e6ffff640 1 -- 172.21.15.33:0/1219407402 learned_addr learned my addr 172.21.15.33:0/1219407402 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e767fc640 1 -- 172.21.15.33:0/1219407402 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1450996761 0 0) 0x7f1e60001120 con 0x7f1e7815e660 2024-04-25T01:09:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e767fc640 1 -- 172.21.15.33:0/1219407402 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e64003530 con 0x7f1e7815e660 2024-04-25T01:09:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e76ffd640 1 -- 172.21.15.33:0/1219407402 >> v1:172.21.15.138:6789/0 conn(0x7f1e78072540 legacy=0x7f1e780ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e76ffd640 1 -- 172.21.15.33:0/1219407402 >> v1:172.21.15.33:6789/0 conn(0x7f1e7815e660 legacy=0x7f1e78071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.156+0000 7f1e76ffd640 1 -- 172.21.15.33:0/1219407402 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7815ec20 con 0x7f1e781444f0 2024-04-25T01:09:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e76ffd640 1 --2- 172.21.15.33:0/1219407402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e781444f0 0x7f1e780b5620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1e680097c0 tx=0x7f1e6801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fec752cda509a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e767fc640 1 -- 172.21.15.33:0/1219407402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680b0070 con 0x7f1e781444f0 2024-04-25T01:09:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e767fc640 1 -- 172.21.15.33:0/1219407402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e680ab600 con 0x7f1e781444f0 2024-04-25T01:09:34.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e767fc640 1 -- 172.21.15.33:0/1219407402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e680ab9d0 con 0x7f1e781444f0 2024-04-25T01:09:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/1219407402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e781444f0 msgr2=0x7f1e780b5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/1219407402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e781444f0 0x7f1e780b5620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1e680097c0 tx=0x7f1e6801c410 comp rx=0 tx=0).stop 2024-04-25T01:09:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/1219407402 shutdown_connections 2024-04-25T01:09:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/1219407402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e781444f0 0x7f1e780b5620 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/1219407402 >> 172.21.15.33:0/1219407402 conn(0x7f1e780b1810 msgr2=0x7f1e780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/1219407402 shutdown_connections 2024-04-25T01:09:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/1219407402 wait complete. 2024-04-25T01:09:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 Processor -- start 2024-04-25T01:09:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- start start 2024-04-25T01:09:34.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e78072540 0x7f1e7815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e781444f0 0x7f1e78154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7815e660 0x7f1e78157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780b37a0 con 0x7f1e7815e660 2024-04-25T01:09:34.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780b3620 con 0x7f1e78072540 2024-04-25T01:09:34.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1e78168a10 con 0x7f1e781444f0 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7815e660 0x7f1e78157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7815e660 0x7f1e78157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34886/0 (socket says 172.21.15.33:34886) 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 -- 172.21.15.33:0/2973726155 learned_addr learned my addr 172.21.15.33:0/2973726155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 -- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e781444f0 msgr2=0x7f1e78154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e781444f0 0x7f1e78154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 -- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e78072540 msgr2=0x7f1e7815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e78072540 0x7f1e7815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 -- 172.21.15.33:0/2973726155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7813e720 con 0x7f1e7815e660 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e777fe640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7815e660 0x7f1e78157970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000c700 tx=0x7f1e7000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e6f7fe640 1 -- 172.21.15.33:0/2973726155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70003b80 con 0x7f1e7815e660 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e6f7fe640 1 -- 172.21.15.33:0/2973726155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e70003d20 con 0x7f1e7815e660 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7813ea00 con 0x7f1e7815e660 2024-04-25T01:09:34.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e6f7fe640 1 -- 172.21.15.33:0/2973726155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000aa90 con 0x7f1e7815e660 2024-04-25T01:09:34.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.160+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7813efc0 con 0x7f1e7815e660 2024-04-25T01:09:34.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.164+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e78069040 con 0x7f1e7815e660 2024-04-25T01:09:34.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.164+0000 7f1e6f7fe640 1 -- 172.21.15.33:0/2973726155 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1e7000ac30 con 0x7f1e7815e660 2024-04-25T01:09:34.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.164+0000 7f1e6f7fe640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1e580678c0 0x7f1e58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.164+0000 7f1e6f7fe640 1 -- 172.21.15.33:0/2973726155 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1e70086040 con 0x7f1e7815e660 2024-04-25T01:09:34.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.164+0000 7f1e76ffd640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1e580678c0 0x7f1e58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.164+0000 7f1e76ffd640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1e580678c0 0x7f1e58069d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1e680ad000 tx=0x7f1e68008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.168+0000 7f1e6f7fe640 1 -- 172.21.15.33:0/2973726155 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1e700566e0 con 0x7f1e7815e660 2024-04-25T01:09:34.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.324+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e78143510 con 0x7f1e7815e660 2024-04-25T01:09:34.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.324+0000 7f1e6f7fe640 1 -- 172.21.15.33:0/2973726155 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18890 (secure 0 0 0) 0x7f1e70055fe0 con 0x7f1e7815e660 2024-04-25T01:09:34.319 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:34.319 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:12.316111+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:08:21.975016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T01:08:22.806669+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T01:08:26.528967+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.b","mappings":[{"from":3,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]},{"pgid":"4.3","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1e580678c0 msgr2=0x7f1e58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1e580678c0 0x7f1e58069d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1e680ad000 tx=0x7f1e68008040 comp rx=0 tx=0).stop 2024-04-25T01:09:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7815e660 msgr2=0x7f1e78157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7815e660 0x7f1e78157970 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000c700 tx=0x7f1e7000cbc0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 shutdown_connections 2024-04-25T01:09:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1e580678c0 0x7f1e58069d80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e7815e660 0x7f1e78157970 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1e781444f0 0x7f1e78154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 --2- 172.21.15.33:0/2973726155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e78072540 0x7f1e7815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 >> 172.21.15.33:0/2973726155 conn(0x7f1e780b1810 msgr2=0x7f1e78071350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 shutdown_connections 2024-04-25T01:09:34.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.328+0000 7f1e7d7ba640 1 -- 172.21.15.33:0/2973726155 wait complete. 2024-04-25T01:09:34.337 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T01:09:34.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.456+0000 7f0c9c8e7640 1 Processor -- start 2024-04-25T01:09:34.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.456+0000 7f0c9c8e7640 1 -- start start 2024-04-25T01:09:34.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f0c9806b920 0x7f0c9815c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c980d0a00 con 0x7f0c9815c2d0 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9815cdc0 con 0x7f0c980d0620 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f0c9806e3a0 con 0x7f0c9806b920 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c95d9b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f0c9815c2d0 0x7f0c9806b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39232/0 (socket says 172.21.15.33:39232) 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c95d9b640 1 -- 172.21.15.33:0/2083296947 learned_addr learned my addr 172.21.15.33:0/2083296947 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c96d9d640 1 --2- 172.21.15.33:0/2083296947 >> v2:172.21.15.33:3300/0 conn(0x7f0c9806b920 0x7f0c9815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9559a640 1 -- 172.21.15.33:0/2083296947 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749217762 0 0) 0x7f0c980d0a00 con 0x7f0c9815c2d0 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9559a640 1 -- 172.21.15.33:0/2083296947 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c80003530 con 0x7f0c9815c2d0 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c96d9d640 1 -- 172.21.15.33:0/2083296947 >> v1:172.21.15.138:6789/0 conn(0x7f0c980d0620 legacy=0x7f0c9806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c96d9d640 1 -- 172.21.15.33:0/2083296947 >> v1:172.21.15.33:6789/0 conn(0x7f0c9815c2d0 legacy=0x7f0c9806b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c96d9d640 1 -- 172.21.15.33:0/2083296947 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9815d0a0 con 0x7f0c9806b920 2024-04-25T01:09:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c96d9d640 1 --2- 172.21.15.33:0/2083296947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c9806b920 0x7f0c9815c760 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009d90 tx=0x7f0c9001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2462f9e92a7f20f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9559a640 1 -- 172.21.15.33:0/2083296947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c900b2070 con 0x7f0c9806b920 2024-04-25T01:09:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9559a640 1 -- 172.21.15.33:0/2083296947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c900a5a00 con 0x7f0c9806b920 2024-04-25T01:09:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9559a640 1 -- 172.21.15.33:0/2083296947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c900a5dd0 con 0x7f0c9806b920 2024-04-25T01:09:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/2083296947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c9806b920 msgr2=0x7f0c9815c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/2083296947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c9806b920 0x7f0c9815c760 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009d90 tx=0x7f0c9001c940 comp rx=0 tx=0).stop 2024-04-25T01:09:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/2083296947 shutdown_connections 2024-04-25T01:09:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/2083296947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c9806b920 0x7f0c9815c760 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/2083296947 >> 172.21.15.33:0/2083296947 conn(0x7f0c980b29c0 msgr2=0x7f0c980b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/2083296947 shutdown_connections 2024-04-25T01:09:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.460+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/2083296947 wait complete. 2024-04-25T01:09:34.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 Processor -- start 2024-04-25T01:09:34.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 -- start start 2024-04-25T01:09:34.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c9806b920 0x7f0c981477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c980d0620 0x7f0c98148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c9815c2d0 0x7f0c9814e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9815de20 con 0x7f0c980d0620 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9815dca0 con 0x7f0c9815c2d0 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9815dfa0 con 0x7f0c9806b920 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c980d0620 0x7f0c98148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c980d0620 0x7f0c98148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34902/0 (socket says 172.21.15.33:34902) 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 -- 172.21.15.33:0/665313719 learned_addr learned my addr 172.21.15.33:0/665313719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 -- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c9806b920 msgr2=0x7f0c981477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c9806b920 0x7f0c981477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 -- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c9815c2d0 msgr2=0x7f0c9814e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c9815c2d0 0x7f0c9814e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 -- 172.21.15.33:0/665313719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c981489c0 con 0x7f0c980d0620 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c95d9b640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c980d0620 0x7f0c98148070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c450 tx=0x7f0c8c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c86ffd640 1 -- 172.21.15.33:0/665313719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c017070 con 0x7f0c980d0620 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c86ffd640 1 -- 172.21.15.33:0/665313719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c8c005cf0 con 0x7f0c980d0620 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c86ffd640 1 -- 172.21.15.33:0/665313719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c007270 con 0x7f0c980d0620 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c98148ca0 con 0x7f0c980d0620 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c98149260 con 0x7f0c980d0620 2024-04-25T01:09:34.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c98069040 con 0x7f0c980d0620 2024-04-25T01:09:34.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c86ffd640 1 -- 172.21.15.33:0/665313719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0c8c01a070 con 0x7f0c980d0620 2024-04-25T01:09:34.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c86ffd640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c780678c0 0x7f0c78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.464+0000 7f0c86ffd640 1 -- 172.21.15.33:0/665313719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0c8c085250 con 0x7f0c980d0620 2024-04-25T01:09:34.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.468+0000 7f0c9659c640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c780678c0 0x7f0c78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.468+0000 7f0c9659c640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c780678c0 0x7f0c78069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000f70 tx=0x7f0c880079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.472+0000 7f0c86ffd640 1 -- 172.21.15.33:0/665313719 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0c8c055340 con 0x7f0c980d0620 2024-04-25T01:09:34.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.660+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0c98073df0 con 0x7f0c980d0620 2024-04-25T01:09:34.652 INFO:tasks.ceph.mon.a.smithi033.stderr:2024-04-25T01:09:34.660+0000 7fad5470a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:09:34.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.660+0000 7f0c86ffd640 1 -- 172.21.15.33:0/665313719 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f0c8c059640 con 0x7f0c980d0620 2024-04-25T01:09:34.654 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T01:09:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c780678c0 msgr2=0x7f0c78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c780678c0 0x7f0c78069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0c88000f70 tx=0x7f0c880079e0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c980d0620 msgr2=0x7f0c98148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c980d0620 0x7f0c98148070 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c450 tx=0x7f0c8c00c910 comp rx=0 tx=0).stop 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 shutdown_connections 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f0c780678c0 0x7f0c78069d80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c9815c2d0 0x7f0c9814e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c980d0620 0x7f0c98148070 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 --2- 172.21.15.33:0/665313719 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f0c9806b920 0x7f0c981477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 >> 172.21.15.33:0/665313719 conn(0x7f0c980b29c0 msgr2=0x7f0c9809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 shutdown_connections 2024-04-25T01:09:34.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.664+0000 7f0c9c8e7640 1 -- 172.21.15.33:0/665313719 wait complete. 2024-04-25T01:09:34.666 INFO:tasks.ceph.mds.c.smithi033.stderr: -4934> 2024-04-25T01:08:27.369+0000 7f3087f47640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:09:34.674 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T01:09:34.724 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2024-04-25T01:09:34.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.800+0000 7f60c21de640 1 Processor -- start 2024-04-25T01:09:34.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.800+0000 7f60c21de640 1 -- start start 2024-04-25T01:09:34.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.800+0000 7f60c21de640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f60bc15ae20 0x7f60bc0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60c21de640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60bc0b4a20 con 0x7f60bc06ecc0 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60c21de640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60bc0b2bb0 con 0x7f60bc072410 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60c21de640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f60bc0b2e90 con 0x7f60bc15ae20 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bbfff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f60bc06ecc0 0x7f60bc0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39234/0 (socket says 172.21.15.33:39234) 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bbfff640 1 -- 172.21.15.33:0/2811519389 learned_addr learned my addr 172.21.15.33:0/2811519389 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/2811519389 >> v2:172.21.15.138:3300/0 conn(0x7f60bc15ae20 0x7f60bc0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246965221 0 0) 0x7f60bc0b4a20 con 0x7f60bc06ecc0 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f60a4003530 con 0x7f60bc06ecc0 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2435116710 0 0) 0x7f60a4003530 con 0x7f60bc06ecc0 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 >> v2:172.21.15.138:3300/0 conn(0x7f60bc15ae20 msgr2=0x7f60bc0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 --2- 172.21.15.33:0/2811519389 >> v2:172.21.15.138:3300/0 conn(0x7f60bc15ae20 0x7f60bc0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 >> v1:172.21.15.138:6789/0 conn(0x7f60bc072410 legacy=0x7f60bc06e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc0b3170 con 0x7f60bc06ecc0 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f60b4001a00 con 0x7f60bc06ecc0 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 >> v1:172.21.15.33:6789/0 conn(0x7f60bc06ecc0 legacy=0x7f60bc0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60a40039c0 0x7f60a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a4004810 0x7f60a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a40051d0 0x7f60a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0b2e90 con 0x7f60a40039c0 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60a400c070 con 0x7f60a40051d0 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60a400c270 con 0x7f60a4004810 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60a40039c0 0x7f60a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bbfff640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a40051d0 0x7f60a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 -- 172.21.15.33:0/2811519389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a4004810 msgr2=0x7f60a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:34.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a4004810 0x7f60a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 -- 172.21.15.33:0/2811519389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a40051d0 msgr2=0x7f60a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:34.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60a40051d0 0x7f60a400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 -- 172.21.15.33:0/2811519389 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a4003530 con 0x7f60a40039c0 2024-04-25T01:09:34.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bbfff640 1 -- 172.21.15.33:0/2811519389 reap_dead start 2024-04-25T01:09:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60a40039c0 0x7f60a4003de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009ae0 tx=0x7f60ac01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488df26b9c166698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0b2070 con 0x7f60a40039c0 2024-04-25T01:09:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60ac0a56f0 con 0x7f60a40039c0 2024-04-25T01:09:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.804+0000 7f60ba7fc640 1 -- 172.21.15.33:0/2811519389 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0096b0 con 0x7f60a40039c0 2024-04-25T01:09:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- 172.21.15.33:0/2811519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60a40039c0 msgr2=0x7f60a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60a40039c0 0x7f60a4003de0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009ae0 tx=0x7f60ac01ca70 comp rx=0 tx=0).stop 2024-04-25T01:09:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- 172.21.15.33:0/2811519389 shutdown_connections 2024-04-25T01:09:34.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 --2- 172.21.15.33:0/2811519389 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60a40039c0 0x7f60a4003de0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- 172.21.15.33:0/2811519389 >> 172.21.15.33:0/2811519389 conn(0x7f60bc0b1810 msgr2=0x7f60bc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:34.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- 172.21.15.33:0/2811519389 shutdown_connections 2024-04-25T01:09:34.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- 172.21.15.33:0/2811519389 wait complete. 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 Processor -- start 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- start start 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a40039c0 0x7f60bc15b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60bc1459b0 0x7f60bc1424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60bc0aebf0 0x7f60bc144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0b44c0 con 0x7f60bc1459b0 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f60bc0b4340 con 0x7f60bc0aebf0 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f60bc0b4640 con 0x7f60a40039c0 2024-04-25T01:09:34.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60bc1459b0 0x7f60bc1424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60bc1459b0 0x7f60bc1424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34928/0 (socket says 172.21.15.33:34928) 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 -- 172.21.15.33:0/3780659290 learned_addr learned my addr 172.21.15.33:0/3780659290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60bbfff640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60bc0aebf0 0x7f60bc144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a40039c0 0x7f60bc15b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 -- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a40039c0 msgr2=0x7f60bc15b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a40039c0 0x7f60bc15b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 -- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60bc0aebf0 msgr2=0x7f60bc144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60bc0aebf0 0x7f60bc144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 -- 172.21.15.33:0/3780659290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60bc142a00 con 0x7f60bc1459b0 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a40039c0 0x7f60bc15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60baffd640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60bc1459b0 0x7f60bc1424c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f60b000c4d0 tx=0x7f60b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60a3fff640 1 -- 172.21.15.33:0/3780659290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b00052d0 con 0x7f60bc1459b0 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60a3fff640 1 -- 172.21.15.33:0/3780659290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60b0005470 con 0x7f60bc1459b0 2024-04-25T01:09:34.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60bc155ef0 con 0x7f60bc1459b0 2024-04-25T01:09:34.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60a3fff640 1 -- 172.21.15.33:0/3780659290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60b0012870 con 0x7f60bc1459b0 2024-04-25T01:09:34.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.808+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60bc156470 con 0x7f60bc1459b0 2024-04-25T01:09:34.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.812+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60bc069040 con 0x7f60bc1459b0 2024-04-25T01:09:34.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.812+0000 7f60a3fff640 1 -- 172.21.15.33:0/3780659290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f60b0012a10 con 0x7f60bc1459b0 2024-04-25T01:09:34.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.812+0000 7f60a3fff640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f60a80678c0 0x7f60a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:34.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.812+0000 7f60a3fff640 1 -- 172.21.15.33:0/3780659290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f60b0085d50 con 0x7f60bc1459b0 2024-04-25T01:09:34.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.812+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f60a80678c0 0x7f60a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:34.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.812+0000 7f60bb7fe640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f60a80678c0 0x7f60a8069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0af000 tx=0x7f60ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:34.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:34.816+0000 7f60a3fff640 1 -- 172.21.15.33:0/3780659290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f60b00182b0 con 0x7f60bc1459b0 2024-04-25T01:09:35.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.020+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f60bc156670 con 0x7f60bc1459b0 2024-04-25T01:09:35.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.664+0000 7f60a3fff640 1 -- 172.21.15.33:0/3780659290 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f60b0056420 con 0x7f60bc1459b0 2024-04-25T01:09:35.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f60a80678c0 msgr2=0x7f60a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f60a80678c0 0x7f60a8069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f60ac0af000 tx=0x7f60ac008040 comp rx=0 tx=0).stop 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60bc1459b0 msgr2=0x7f60bc1424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60bc1459b0 0x7f60bc1424c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f60b000c4d0 tx=0x7f60b000c990 comp rx=0 tx=0).stop 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 shutdown_connections 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f60a80678c0 0x7f60a8069d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60bc0aebf0 0x7f60bc144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60bc1459b0 0x7f60bc1424c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 --2- 172.21.15.33:0/3780659290 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f60a40039c0 0x7f60bc15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 >> 172.21.15.33:0/3780659290 conn(0x7f60bc0b1810 msgr2=0x7f60bc06dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 shutdown_connections 2024-04-25T01:09:35.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.668+0000 7f60c21de640 1 -- 172.21.15.33:0/3780659290 wait complete. 2024-04-25T01:09:35.682 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T01:09:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d94bfa640 1 Processor -- start 2024-04-25T01:09:35.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d94bfa640 1 -- start start 2024-04-25T01:09:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d94bfa640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d901457d0 0x7f6d9006e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d94bfa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 0x7f6d9015ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d94bfa640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d900b2b40 con 0x7f6d901453f0 2024-04-25T01:09:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d94bfa640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6d900b3430 con 0x7f6d901457d0 2024-04-25T01:09:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d94bfa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d900b2de0 con 0x7f6d900d0320 2024-04-25T01:09:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d8e59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6d901453f0 0x7f6d900cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39248/0 (socket says 172.21.15.33:39248) 2024-04-25T01:09:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d8e59c640 1 -- 172.21.15.33:0/3579425539 learned_addr learned my addr 172.21.15.33:0/3579425539 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4047985341 0 0) 0x7f6d900b2b40 con 0x7f6d901453f0 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d7c003530 con 0x7f6d901453f0 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.816+0000 7f6d8dd9b640 1 --2- 172.21.15.33:0/3579425539 >> v2:172.21.15.138:3300/0 conn(0x7f6d901457d0 0x7f6d9006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 0x7f6d9015ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1925080710 0 0) 0x7f6d7c003530 con 0x7f6d901453f0 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 msgr2=0x7f6d9015ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 0x7f6d9015ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 >> v2:172.21.15.138:3300/0 conn(0x7f6d901457d0 msgr2=0x7f6d9006e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 --2- 172.21.15.33:0/3579425539 >> v2:172.21.15.138:3300/0 conn(0x7f6d901457d0 0x7f6d9006e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d900b30c0 con 0x7f6d901453f0 2024-04-25T01:09:35.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f6d80001a20 con 0x7f6d901453f0 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 >> v1:172.21.15.33:6789/0 conn(0x7f6d901453f0 legacy=0x7f6d900cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 0x7f6d7c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c003f90 0x7f6d7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c004d50 0x7f6d7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900b3430 con 0x7f6d7c004d50 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900b2de0 con 0x7f6d7c003f90 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d7c00ba80 con 0x7f6d900d0320 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c004d50 0x7f6d7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8e59c640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 0x7f6d7c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8dd9b640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c003f90 0x7f6d7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 msgr2=0x7f6d7c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900d0320 0x7f6d7c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c003f90 msgr2=0x7f6d7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c003f90 0x7f6d7c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/3579425539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c003530 con 0x7f6d7c004d50 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c004d50 0x7f6d7c0051d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800ba10 tx=0x7f6d8801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3814efc2ecf21df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d880ae070 con 0x7f6d7c004d50 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d880a9660 con 0x7f6d7c004d50 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8d59a640 1 -- 172.21.15.33:0/3579425539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800b610 con 0x7f6d7c004d50 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d94bfa640 1 -- 172.21.15.33:0/3579425539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c004d50 msgr2=0x7f6d7c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d94bfa640 1 --2- 172.21.15.33:0/3579425539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d7c004d50 0x7f6d7c0051d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6d8800ba10 tx=0x7f6d8801c680 comp rx=0 tx=0).stop 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/3579425539 reap_dead start 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d94bfa640 1 -- 172.21.15.33:0/3579425539 shutdown_connections 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d94bfa640 1 -- 172.21.15.33:0/3579425539 >> 172.21.15.33:0/3579425539 conn(0x7f6d9006b3e0 msgr2=0x7f6d9006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:35.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d94bfa640 1 -- 172.21.15.33:0/3579425539 shutdown_connections 2024-04-25T01:09:35.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.820+0000 7f6d94bfa640 1 -- 172.21.15.33:0/3579425539 wait complete. 2024-04-25T01:09:35.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 Processor -- start 2024-04-25T01:09:35.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 -- start start 2024-04-25T01:09:35.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c004d50 0x7f6d9013eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900b0990 0x7f6d901f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d901f4f20 0x7f6d901f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900b3a40 con 0x7f6d901f4f20 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d900b38c0 con 0x7f6d7c004d50 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d9015ce50 con 0x7f6d900b0990 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d901f4f20 0x7f6d901f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d901f4f20 0x7f6d901f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34946/0 (socket says 172.21.15.33:34946) 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/2915352792 learned_addr learned my addr 172.21.15.33:0/2915352792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8e59c640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c004d50 0x7f6d9013eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900b0990 msgr2=0x7f6d901f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900b0990 0x7f6d901f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c004d50 msgr2=0x7f6d9013eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c004d50 0x7f6d9013eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 -- 172.21.15.33:0/2915352792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d90143c30 con 0x7f6d901f4f20 2024-04-25T01:09:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d8ed9d640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d901f4f20 0x7f6d901f53a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6d8801cf20 tx=0x7f6d7c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d86ffd640 1 -- 172.21.15.33:0/2915352792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d880ae040 con 0x7f6d901f4f20 2024-04-25T01:09:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d86ffd640 1 -- 172.21.15.33:0/2915352792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d880b7070 con 0x7f6d901f4f20 2024-04-25T01:09:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d86ffd640 1 -- 172.21.15.33:0/2915352792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8800b610 con 0x7f6d901f4f20 2024-04-25T01:09:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9013f400 con 0x7f6d901f4f20 2024-04-25T01:09:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d9013f9c0 con 0x7f6d901f4f20 2024-04-25T01:09:35.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.824+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d90069040 con 0x7f6d901f4f20 2024-04-25T01:09:35.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.828+0000 7f6d86ffd640 1 -- 172.21.15.33:0/2915352792 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d880b1070 con 0x7f6d901f4f20 2024-04-25T01:09:35.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.828+0000 7f6d86ffd640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d700678f0 0x7f6d70069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:35.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.828+0000 7f6d86ffd640 1 -- 172.21.15.33:0/2915352792 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f6d88120c70 con 0x7f6d901f4f20 2024-04-25T01:09:35.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.828+0000 7f6d8e59c640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d700678f0 0x7f6d70069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:35.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.828+0000 7f6d8e59c640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d700678f0 0x7f6d70069db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d900b1580 tx=0x7f6d80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:35.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.832+0000 7f6d86ffd640 1 -- 172.21.15.33:0/2915352792 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6d88016390 con 0x7f6d901f4f20 2024-04-25T01:09:35.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:35.996+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6d900b48a0 con 0x7f6d901f4f20 2024-04-25T01:09:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.672+0000 7f6d86ffd640 1 -- 172.21.15.33:0/2915352792 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f6d880f13e0 con 0x7f6d901f4f20 2024-04-25T01:09:36.666 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-04-25T01:09:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d700678f0 msgr2=0x7f6d70069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d700678f0 0x7f6d70069db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d900b1580 tx=0x7f6d80009040 comp rx=0 tx=0).stop 2024-04-25T01:09:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d901f4f20 msgr2=0x7f6d901f53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d901f4f20 0x7f6d901f53a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6d8801cf20 tx=0x7f6d7c005840 comp rx=0 tx=0).stop 2024-04-25T01:09:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 shutdown_connections 2024-04-25T01:09:36.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d700678f0 0x7f6d70069db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d901f4f20 0x7f6d901f53a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d900b0990 0x7f6d901f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 --2- 172.21.15.33:0/2915352792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d7c004d50 0x7f6d9013eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 >> 172.21.15.33:0/2915352792 conn(0x7f6d9006b3e0 msgr2=0x7f6d900cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:36.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.676+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 shutdown_connections 2024-04-25T01:09:36.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.680+0000 7f6d94bfa640 1 -- 172.21.15.33:0/2915352792 wait complete. 2024-04-25T01:09:36.691 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T01:09:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.824+0000 7f0036546640 1 Processor -- start 2024-04-25T01:09:36.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.824+0000 7f0036546640 1 -- start start 2024-04-25T01:09:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f0036546640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00300cff90 0x7f0030070460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f0036546640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00300ceef0 con 0x7f003006b210 2024-04-25T01:09:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f0036546640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00300cf1d0 con 0x7f0030144860 2024-04-25T01:09:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f0036546640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f00300cf4b0 con 0x7f00300cff90 2024-04-25T01:09:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f002f7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f00300cff90 0x7f0030070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f0034ae3640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f003006b210 0x7f00300ad840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39256/0 (socket says 172.21.15.33:39256) 2024-04-25T01:09:36.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f0034ae3640 1 -- 172.21.15.33:0/857275709 learned_addr learned my addr 172.21.15.33:0/857275709 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f002f7fe640 1 -- 172.21.15.33:0/857275709 >> v1:172.21.15.138:6789/0 conn(0x7f0030144860 legacy=0x7f00300ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f002f7fe640 1 -- 172.21.15.33:0/857275709 >> v1:172.21.15.33:6789/0 conn(0x7f003006b210 legacy=0x7f00300ad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f002f7fe640 1 -- 172.21.15.33:0/857275709 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00300cf790 con 0x7f00300cff90 2024-04-25T01:09:36.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f002effd640 1 -- 172.21.15.33:0/857275709 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666208085 0 0) 0x7f00300ceef0 con 0x7f003006b210 2024-04-25T01:09:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.828+0000 7f002f7fe640 1 --2- 172.21.15.33:0/857275709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00300cff90 0x7f0030070460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f00240097c0 tx=0x7f002401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6db6c5683f9b01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002effd640 1 -- 172.21.15.33:0/857275709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00240b0070 con 0x7f00300cff90 2024-04-25T01:09:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002effd640 1 -- 172.21.15.33:0/857275709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00240ab600 con 0x7f00300cff90 2024-04-25T01:09:36.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002effd640 1 -- 172.21.15.33:0/857275709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00240ab9d0 con 0x7f00300cff90 2024-04-25T01:09:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- 172.21.15.33:0/857275709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00300cff90 msgr2=0x7f0030070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 --2- 172.21.15.33:0/857275709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00300cff90 0x7f0030070460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f00240097c0 tx=0x7f002401c3d0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- 172.21.15.33:0/857275709 shutdown_connections 2024-04-25T01:09:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 --2- 172.21.15.33:0/857275709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00300cff90 0x7f0030070460 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- 172.21.15.33:0/857275709 >> 172.21.15.33:0/857275709 conn(0x7f00300d0af0 msgr2=0x7f00300d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:36.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- 172.21.15.33:0/857275709 shutdown_connections 2024-04-25T01:09:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- 172.21.15.33:0/857275709 wait complete. 2024-04-25T01:09:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 Processor -- start 2024-04-25T01:09:36.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- start start 2024-04-25T01:09:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003006b210 0x7f003013ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00300cff90 0x7f0030160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030144860 0x7f0030160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f003016bf40 con 0x7f0030144860 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f003016bdc0 con 0x7f003006b210 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f003016c0c0 con 0x7f00300cff90 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030144860 0x7f0030160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030144860 0x7f0030160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34964/0 (socket says 172.21.15.33:34964) 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 -- 172.21.15.33:0/1188235397 learned_addr learned my addr 172.21.15.33:0/1188235397 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002f7fe640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00300cff90 0x7f0030160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 -- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00300cff90 msgr2=0x7f0030160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00300cff90 0x7f0030160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 -- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003006b210 msgr2=0x7f003013ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002ffff640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003006b210 0x7f003013ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003006b210 0x7f003013ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 -- 172.21.15.33:0/1188235397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003013d390 con 0x7f0030144860 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002ffff640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003006b210 0x7f003013ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0034ae3640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030144860 0x7f0030160a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f002800c4b0 tx=0x7f002800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002cff9640 1 -- 172.21.15.33:0/1188235397 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028018070 con 0x7f0030144860 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002cff9640 1 -- 172.21.15.33:0/1188235397 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00280074e0 con 0x7f0030144860 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0030161050 con 0x7f0030144860 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f002cff9640 1 -- 172.21.15.33:0/1188235397 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0028004a30 con 0x7f0030144860 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.832+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0030144e90 con 0x7f0030144860 2024-04-25T01:09:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.836+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0030069040 con 0x7f0030144860 2024-04-25T01:09:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.836+0000 7f002cff9640 1 -- 172.21.15.33:0/1188235397 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f002801b070 con 0x7f0030144860 2024-04-25T01:09:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.836+0000 7f002cff9640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f001c0678c0 0x7f001c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.836+0000 7f002cff9640 1 -- 172.21.15.33:0/1188235397 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7f00280851f0 con 0x7f0030144860 2024-04-25T01:09:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.836+0000 7f002ffff640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f001c0678c0 0x7f001c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.836+0000 7f002ffff640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f001c0678c0 0x7f001c069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0020007e90 tx=0x7f00200073a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:36.840+0000 7f002cff9640 1 -- 172.21.15.33:0/1188235397 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f0028055a90 con 0x7f0030144860 2024-04-25T01:09:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.008+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f00300adbc0 con 0x7f0030144860 2024-04-25T01:09:37.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.680+0000 7f002cff9640 1 -- 172.21.15.33:0/1188235397 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f0028055390 con 0x7f0030144860 2024-04-25T01:09:37.674 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-04-25T01:09:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f001c0678c0 msgr2=0x7f001c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f001c0678c0 0x7f001c069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0020007e90 tx=0x7f00200073a0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030144860 msgr2=0x7f0030160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:37.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030144860 0x7f0030160a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f002800c4b0 tx=0x7f002800c970 comp rx=0 tx=0).stop 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 shutdown_connections 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f001c0678c0 0x7f001c069d80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0030144860 0x7f0030160a40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00300cff90 0x7f0030160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 --2- 172.21.15.33:0/1188235397 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f003006b210 0x7f003013ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 >> 172.21.15.33:0/1188235397 conn(0x7f00300d0af0 msgr2=0x7f00300af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 shutdown_connections 2024-04-25T01:09:37.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.688+0000 7f0036546640 1 -- 172.21.15.33:0/1188235397 wait complete. 2024-04-25T01:09:37.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T01:09:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff854f66640 1 Processor -- start 2024-04-25T01:09:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff854f66640 1 -- start start 2024-04-25T01:09:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff854f66640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff8501625a0 0x7ff8500d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff854f66640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff85015e4a0 0x7ff8500d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff854f66640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8500b0fe0 con 0x7ff85015e0c0 2024-04-25T01:09:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff854f66640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ff8500d0e20 con 0x7ff8501625a0 2024-04-25T01:09:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff854f66640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff8500d1100 con 0x7ff85015e4a0 2024-04-25T01:09:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff84e59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff85015e0c0 0x7ff850161e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39264/0 (socket says 172.21.15.33:39264) 2024-04-25T01:09:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.836+0000 7ff84e59c640 1 -- 172.21.15.33:0/437384132 learned_addr learned my addr 172.21.15.33:0/437384132 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 --2- 172.21.15.33:0/437384132 >> v2:172.21.15.138:3300/0 conn(0x7ff8501625a0 0x7ff8500d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84ed9d640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff85015e4a0 0x7ff8500d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1776992436 0 0) 0x7ff840001120 con 0x7ff85015e0c0 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff838003530 con 0x7ff85015e0c0 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3611380282 0 0) 0x7ff838003530 con 0x7ff85015e0c0 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff85015e4a0 msgr2=0x7ff8500d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff85015e4a0 0x7ff8500d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 >> v2:172.21.15.138:3300/0 conn(0x7ff8501625a0 msgr2=0x7ff8500d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 --2- 172.21.15.33:0/437384132 >> v2:172.21.15.138:3300/0 conn(0x7ff8501625a0 0x7ff8500d0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8500aec60 con 0x7ff85015e0c0 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7ff840001850 con 0x7ff85015e0c0 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 >> v1:172.21.15.33:6789/0 conn(0x7ff85015e0c0 legacy=0x7ff850161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8380039c0 0x7ff838003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838004810 0x7ff838004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff85015e4a0 0x7ff8380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500d0e20 con 0x7ff838004810 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500d1100 con 0x7ff8380039c0 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff83800b9d0 con 0x7ff85015e4a0 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838004810 0x7ff838004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84e59c640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8380039c0 0x7ff838003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 -- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff85015e4a0 msgr2=0x7ff8380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:37.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff85015e4a0 0x7ff8380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 -- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8380039c0 msgr2=0x7ff838003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:37.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8380039c0 0x7ff838003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 -- 172.21.15.33:0/437384132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff838003530 con 0x7ff838004810 2024-04-25T01:09:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84dd9b640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838004810 0x7ff838004c90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff844009ae0 tx=0x7ff84401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d132155b1980aee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8440b2070 con 0x7ff838004810 2024-04-25T01:09:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8440a55c0 con 0x7ff838004810 2024-04-25T01:09:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84d59a640 1 -- 172.21.15.33:0/437384132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8440096b0 con 0x7ff838004810 2024-04-25T01:09:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff854f66640 1 -- 172.21.15.33:0/437384132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838004810 msgr2=0x7ff838004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff854f66640 1 --2- 172.21.15.33:0/437384132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff838004810 0x7ff838004c90 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff844009ae0 tx=0x7ff84401ca50 comp rx=0 tx=0).stop 2024-04-25T01:09:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff84ed9d640 1 -- 172.21.15.33:0/437384132 reap_dead start 2024-04-25T01:09:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff854f66640 1 -- 172.21.15.33:0/437384132 shutdown_connections 2024-04-25T01:09:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.840+0000 7ff854f66640 1 -- 172.21.15.33:0/437384132 >> 172.21.15.33:0/437384132 conn(0x7ff8500d1ca0 msgr2=0x7ff8500d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- 172.21.15.33:0/437384132 shutdown_connections 2024-04-25T01:09:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- 172.21.15.33:0/437384132 wait complete. 2024-04-25T01:09:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 Processor -- start 2024-04-25T01:09:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- start start 2024-04-25T01:09:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff838004810 0x7ff8501f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850161490 0x7ff8500b7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8500c7440 0x7ff8501f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500d1820 con 0x7ff8500c7440 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8500d16a0 con 0x7ff850161490 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff8500d19a0 con 0x7ff838004810 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8500c7440 0x7ff8501f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8500c7440 0x7ff8501f3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34984/0 (socket says 172.21.15.33:34984) 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 -- 172.21.15.33:0/1147093231 learned_addr learned my addr 172.21.15.33:0/1147093231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84e59c640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff838004810 0x7ff8501f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84dd9b640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850161490 0x7ff8500b7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 -- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff838004810 msgr2=0x7ff8501f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff838004810 0x7ff8501f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 -- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850161490 msgr2=0x7ff8500b7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850161490 0x7ff8500b7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 -- 172.21.15.33:0/1147093231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8500c78c0 con 0x7ff8500c7440 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff84ed9d640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8500c7440 0x7ff8501f3ec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff850071960 tx=0x7ff84800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff83effd640 1 -- 172.21.15.33:0/1147093231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848018020 con 0x7ff8500c7440 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff83effd640 1 -- 172.21.15.33:0/1147093231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff84800bce0 con 0x7ff8500c7440 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8500c7b70 con 0x7ff8500c7440 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff83effd640 1 -- 172.21.15.33:0/1147093231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848014640 con 0x7ff8500c7440 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.844+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8500b4fe0 con 0x7ff8500c7440 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.848+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8500b4af0 con 0x7ff8500c7440 2024-04-25T01:09:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.848+0000 7ff83effd640 1 -- 172.21.15.33:0/1147093231 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff84801b020 con 0x7ff8500c7440 2024-04-25T01:09:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.848+0000 7ff83effd640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff8300678c0 0x7ff830069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.848+0000 7ff83effd640 1 -- 172.21.15.33:0/1147093231 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7ff848086330 con 0x7ff8500c7440 2024-04-25T01:09:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.848+0000 7ff84e59c640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff8300678c0 0x7ff830069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.848+0000 7ff84e59c640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff8300678c0 0x7ff830069d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8500c88b0 tx=0x7ff840001e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:37.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:37.852+0000 7ff83effd640 1 -- 172.21.15.33:0/1147093231 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff848022030 con 0x7ff8500c7440 2024-04-25T01:09:38.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.016+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff850069040 con 0x7ff8500c7440 2024-04-25T01:09:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.692+0000 7ff83effd640 1 -- 172.21.15.33:0/1147093231 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7ff848056df0 con 0x7ff8500c7440 2024-04-25T01:09:38.686 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data_ec' removed 2024-04-25T01:09:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff8300678c0 msgr2=0x7ff830069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff8300678c0 0x7ff830069d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8500c88b0 tx=0x7ff840001e80 comp rx=0 tx=0).stop 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8500c7440 msgr2=0x7ff8501f3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8500c7440 0x7ff8501f3ec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff850071960 tx=0x7ff84800b480 comp rx=0 tx=0).stop 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 shutdown_connections 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff8300678c0 0x7ff830069d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8500c7440 0x7ff8501f3ec0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff850161490 0x7ff8500b7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 --2- 172.21.15.33:0/1147093231 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff838004810 0x7ff8501f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 >> 172.21.15.33:0/1147093231 conn(0x7ff8500d1ca0 msgr2=0x7ff85006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 shutdown_connections 2024-04-25T01:09:38.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.696+0000 7ff854f66640 1 -- 172.21.15.33:0/1147093231 wait complete. 2024-04-25T01:09:38.715 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T01:09:38.716 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T01:09:38.716 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T01:09:38.716 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:38.716 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T01:09:38.716 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T01:09:38.716 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T01:09:38.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:38.717 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T01:09:38.717 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T01:09:38.717 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T01:09:38.717 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:38.717 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T01:09:38.717 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T01:09:38.718 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T01:09:38.718 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:09:38.729 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T01:09:38.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T01:09:38.730 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T01:09:38.730 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T01:09:38.733 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T01:09:38.733 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T01:09:38.736 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T01:09:38.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T01:09:38.772 INFO:tasks.ceph.mds.a:Started 2024-04-25T01:09:38.773 DEBUG:teuthology.parallel:result is None 2024-04-25T01:09:38.773 INFO:tasks.ceph.mds.c:Started 2024-04-25T01:09:38.773 DEBUG:teuthology.parallel:result is None 2024-04-25T01:09:38.782 INFO:tasks.ceph.mds.b:Started 2024-04-25T01:09:38.783 DEBUG:teuthology.parallel:result is None 2024-04-25T01:09:38.784 INFO:tasks.ceph.mds.d:Started 2024-04-25T01:09:38.784 DEBUG:teuthology.parallel:result is None 2024-04-25T01:09:38.784 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T01:09:38.846 INFO:tasks.ceph.mds.a.smithi033.stdout:starting mds.a at 2024-04-25T01:09:38.847 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2024-04-25T01:09:38.854 INFO:tasks.ceph.mds.d.smithi138.stdout:starting mds.d at 2024-04-25T01:09:38.858 INFO:tasks.ceph.mds.b.smithi138.stdout:starting mds.b at 2024-04-25T01:09:38.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.920+0000 7f6d56590640 1 Processor -- start 2024-04-25T01:09:38.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.920+0000 7f6d56590640 1 -- start start 2024-04-25T01:09:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.920+0000 7f6d56590640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6d5015e810 0x7f6d5015eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.920+0000 7f6d56590640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d500b4800 con 0x7f6d5006e270 2024-04-25T01:09:38.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.920+0000 7f6d56590640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d500b4ae0 con 0x7f6d50144730 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.920+0000 7f6d56590640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6d500b4dc0 con 0x7f6d5015e810 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.920+0000 7f6d54b2d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6d5006e270 0x7f6d5006e680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39762/0 (socket says 172.21.15.33:39762) 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d54b2d640 1 -- 172.21.15.33:0/3367547939 learned_addr learned my addr 172.21.15.33:0/3367547939 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4f7fe640 1 --2- 172.21.15.33:0/3367547939 >> v2:172.21.15.138:3300/0 conn(0x7f6d5015e810 0x7f6d5015eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4effd640 1 -- 172.21.15.33:0/3367547939 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1797756789 0 0) 0x7f6d500b4800 con 0x7f6d5006e270 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4effd640 1 -- 172.21.15.33:0/3367547939 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d38003530 con 0x7f6d5006e270 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4f7fe640 1 -- 172.21.15.33:0/3367547939 >> v1:172.21.15.138:6789/0 conn(0x7f6d50144730 legacy=0x7f6d500b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4f7fe640 1 -- 172.21.15.33:0/3367547939 >> v1:172.21.15.33:6789/0 conn(0x7f6d5006e270 legacy=0x7f6d5006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:38.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4f7fe640 1 -- 172.21.15.33:0/3367547939 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5006dd50 con 0x7f6d5015e810 2024-04-25T01:09:38.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4f7fe640 1 --2- 172.21.15.33:0/3367547939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5015e810 0x7f6d5015eca0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097c0 tx=0x7f6d4401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64ab3124b3a3918e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:38.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4effd640 1 -- 172.21.15.33:0/3367547939 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440b1070 con 0x7f6d5015e810 2024-04-25T01:09:38.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4effd640 1 -- 172.21.15.33:0/3367547939 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d440ac410 con 0x7f6d5015e810 2024-04-25T01:09:38.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d4effd640 1 -- 172.21.15.33:0/3367547939 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5006f5a0 con 0x7f6d5015e810 2024-04-25T01:09:38.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d56590640 1 -- 172.21.15.33:0/3367547939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5015e810 msgr2=0x7f6d5015eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:38.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d56590640 1 --2- 172.21.15.33:0/3367547939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5015e810 0x7f6d5015eca0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097c0 tx=0x7f6d4401c4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d56590640 1 -- 172.21.15.33:0/3367547939 shutdown_connections 2024-04-25T01:09:38.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d56590640 1 --2- 172.21.15.33:0/3367547939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5015e810 0x7f6d5015eca0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d56590640 1 -- 172.21.15.33:0/3367547939 >> 172.21.15.33:0/3367547939 conn(0x7f6d500b29c0 msgr2=0x7f6d500b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:38.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d56590640 1 -- 172.21.15.33:0/3367547939 shutdown_connections 2024-04-25T01:09:38.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.924+0000 7f6d56590640 1 -- 172.21.15.33:0/3367547939 wait complete. 2024-04-25T01:09:38.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 Processor -- start 2024-04-25T01:09:38.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 -- start start 2024-04-25T01:09:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5006e270 0x7f6d50142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50144730 0x7f6d50140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5015e810 0x7f6d5013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d50168d70 con 0x7f6d50144730 2024-04-25T01:09:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6d5015f6a0 con 0x7f6d5006e270 2024-04-25T01:09:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6d50168ef0 con 0x7f6d5015e810 2024-04-25T01:09:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50144730 0x7f6d50140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50144730 0x7f6d50140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53604/0 (socket says 172.21.15.33:53604) 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 -- 172.21.15.33:0/2407834407 learned_addr learned my addr 172.21.15.33:0/2407834407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d54b2d640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5015e810 0x7f6d5013e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 -- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5015e810 msgr2=0x7f6d5013e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5015e810 0x7f6d5013e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 -- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5006e270 msgr2=0x7f6d50142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4ffff640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5006e270 0x7f6d50142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5006e270 0x7f6d50142df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 -- 172.21.15.33:0/2407834407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d50141810 con 0x7f6d50144730 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d54b2d640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5015e810 0x7f6d5013e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4ffff640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5006e270 0x7f6d50142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4f7fe640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50144730 0x7f6d50140040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097c0 tx=0x7f6d44019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4cff9640 1 -- 172.21.15.33:0/2407834407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440bc070 con 0x7f6d50144730 2024-04-25T01:09:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4cff9640 1 -- 172.21.15.33:0/2407834407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d440acd70 con 0x7f6d50144730 2024-04-25T01:09:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d50140770 con 0x7f6d50144730 2024-04-25T01:09:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d4cff9640 1 -- 172.21.15.33:0/2407834407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d440b1040 con 0x7f6d50144730 2024-04-25T01:09:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d50140d30 con 0x7f6d50144730 2024-04-25T01:09:38.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.928+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d50069040 con 0x7f6d50144730 2024-04-25T01:09:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.932+0000 7f6d4cff9640 1 -- 172.21.15.33:0/2407834407 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d440bf070 con 0x7f6d50144730 2024-04-25T01:09:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.932+0000 7f6d4cff9640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d3c0678c0 0x7f6d3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.932+0000 7f6d4cff9640 1 -- 172.21.15.33:0/2407834407 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f6d4411eac0 con 0x7f6d50144730 2024-04-25T01:09:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.932+0000 7f6d4ffff640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d3c0678c0 0x7f6d3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:38.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.932+0000 7f6d4ffff640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d3c0678c0 0x7f6d3c069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6d40007fd0 tx=0x7f6d40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:38.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:38.936+0000 7f6d4cff9640 1 -- 172.21.15.33:0/2407834407 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6d50141810 con 0x7f6d50144730 2024-04-25T01:09:39.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.108+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6d50073130 con 0x7f6d50144730 2024-04-25T01:09:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.720+0000 7f6d4cff9640 1 -- 172.21.15.33:0/2407834407 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f6d44016090 con 0x7f6d50144730 2024-04-25T01:09:39.715 INFO:teuthology.orchestra.run.smithi033.stderr: removed all blocklist entries 2024-04-25T01:09:39.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d3c0678c0 msgr2=0x7f6d3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d3c0678c0 0x7f6d3c069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6d40007fd0 tx=0x7f6d40009040 comp rx=0 tx=0).stop 2024-04-25T01:09:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50144730 msgr2=0x7f6d50140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50144730 0x7f6d50140040 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6d440097c0 tx=0x7f6d44019240 comp rx=0 tx=0).stop 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 shutdown_connections 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6d3c0678c0 0x7f6d3c069d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6d5015e810 0x7f6d5013e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d50144730 0x7f6d50140040 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 --2- 172.21.15.33:0/2407834407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d5006e270 0x7f6d50142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 >> 172.21.15.33:0/2407834407 conn(0x7f6d500b29c0 msgr2=0x7f6d5015cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 shutdown_connections 2024-04-25T01:09:39.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.728+0000 7f6d56590640 1 -- 172.21.15.33:0/2407834407 wait complete. 2024-04-25T01:09:39.748 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T01:09:39.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.872+0000 7fccb30c1640 1 Processor -- start 2024-04-25T01:09:39.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb30c1640 1 -- start start 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb30c1640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fccac0d1e20 0x7fccac06c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb30c1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0b1010 0x7fccac0d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb30c1640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccac06e2e0 con 0x7fccac0d2200 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb30c1640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fccac06ddd0 con 0x7fccac0d1e20 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb30c1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccac15e510 con 0x7fccac0b1010 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fccac0d1e20 0x7fccac06c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fccac0d1e20 0x7fccac06c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53608/0 (socket says 172.21.15.33:53608) 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 -- 172.21.15.33:0/3383791872 learned_addr learned my addr 172.21.15.33:0/3383791872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb165e640 1 --1- 172.21.15.33:0/3383791872 >> v1:172.21.15.33:6789/0 conn(0x7fccac0d2200 0x7fccac0b4ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39768/0 (socket says 172.21.15.33:39768) 2024-04-25T01:09:39.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccabfff640 1 --2- 172.21.15.33:0/3383791872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0b1010 0x7fccac0d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:39.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 -- 172.21.15.33:0/3383791872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0b1010 msgr2=0x7fccac0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:39.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 --2- 172.21.15.33:0/3383791872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0b1010 0x7fccac0d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccab7fe640 1 -- 172.21.15.33:0/3383791872 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700276121 0 0) 0x7fccac06e2e0 con 0x7fccac0d2200 2024-04-25T01:09:39.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 -- 172.21.15.33:0/3383791872 >> v1:172.21.15.33:6789/0 conn(0x7fccac0d2200 legacy=0x7fccac0b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:39.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 -- 172.21.15.33:0/3383791872 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac06e000 con 0x7fccac0d1e20 2024-04-25T01:09:39.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccabfff640 1 --2- 172.21.15.33:0/3383791872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0b1010 0x7fccac0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:39.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.876+0000 7fccb0e5d640 1 --2- 172.21.15.33:0/3383791872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac06c320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009870 tx=0x7fcc9c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c9fc0f5462972f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:39.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccab7fe640 1 -- 172.21.15.33:0/3383791872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0b5070 con 0x7fccac0d1e20 2024-04-25T01:09:39.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccab7fe640 1 -- 172.21.15.33:0/3383791872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c0a8a50 con 0x7fccac0d1e20 2024-04-25T01:09:39.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccab7fe640 1 -- 172.21.15.33:0/3383791872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0a8e20 con 0x7fccac0d1e20 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- 172.21.15.33:0/3383791872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 msgr2=0x7fccac06c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 --2- 172.21.15.33:0/3383791872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac06c320 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009870 tx=0x7fcc9c01ed50 comp rx=0 tx=0).stop 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- 172.21.15.33:0/3383791872 shutdown_connections 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 --2- 172.21.15.33:0/3383791872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0b1010 0x7fccac0d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 --2- 172.21.15.33:0/3383791872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac06c320 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- 172.21.15.33:0/3383791872 >> 172.21.15.33:0/3383791872 conn(0x7fccac09a9d0 msgr2=0x7fccac09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- 172.21.15.33:0/3383791872 shutdown_connections 2024-04-25T01:09:39.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- 172.21.15.33:0/3383791872 wait complete. 2024-04-25T01:09:39.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 Processor -- start 2024-04-25T01:09:39.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- start start 2024-04-25T01:09:39.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac0b1010 0x7fccac1460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:39.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac09f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:39.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0d2200 0x7fccac1f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:39.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fccac0d2a00 con 0x7fccac0d1e20 2024-04-25T01:09:39.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fccac0d2880 con 0x7fccac0b1010 2024-04-25T01:09:39.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fccac0d1850 con 0x7fccac0d2200 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac09f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac09f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53612/0 (socket says 172.21.15.33:53612) 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 -- 172.21.15.33:0/273678347 learned_addr learned my addr 172.21.15.33:0/273678347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb165e640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0d2200 0x7fccac1f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 -- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0d2200 msgr2=0x7fccac1f3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0d2200 0x7fccac1f3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb0e5d640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac0b1010 0x7fccac1460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 -- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac0b1010 msgr2=0x7fccac1460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac0b1010 0x7fccac1460d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 -- 172.21.15.33:0/273678347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccac09c0d0 con 0x7fccac0d1e20 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccabfff640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac09f560 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c7c0 tx=0x7fcca000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fcca97fa640 1 -- 172.21.15.33:0/273678347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca0007a30 con 0x7fccac0d1e20 2024-04-25T01:09:39.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fcca97fa640 1 -- 172.21.15.33:0/273678347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca0007bd0 con 0x7fccac0d1e20 2024-04-25T01:09:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccac09c3b0 con 0x7fccac0d1e20 2024-04-25T01:09:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.880+0000 7fcca97fa640 1 -- 172.21.15.33:0/273678347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca00113f0 con 0x7fccac0d1e20 2024-04-25T01:09:39.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.884+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccac09c970 con 0x7fccac0d1e20 2024-04-25T01:09:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.884+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccac069040 con 0x7fccac0d1e20 2024-04-25T01:09:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.884+0000 7fcca97fa640 1 -- 172.21.15.33:0/273678347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcca0011590 con 0x7fccac0d1e20 2024-04-25T01:09:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.884+0000 7fcca97fa640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcc980678c0 0x7fcc98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.884+0000 7fcca97fa640 1 -- 172.21.15.33:0/273678347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fcca0085d60 con 0x7fccac0d1e20 2024-04-25T01:09:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.884+0000 7fccb0e5d640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcc980678c0 0x7fcc98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:39.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.884+0000 7fccb0e5d640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcc980678c0 0x7fcc98069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b2000 tx=0x7fcc9c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:39.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:39.888+0000 7fcca97fa640 1 -- 172.21.15.33:0/273678347 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcca0015020 con 0x7fccac0d1e20 2024-04-25T01:09:40.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.112+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fccac070f40 con 0x7fccac0d1e20 2024-04-25T01:09:40.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.112+0000 7fcca97fa640 1 -- 172.21.15.33:0/273678347 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fcca00569e0 con 0x7fccac0d1e20 2024-04-25T01:09:40.106 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:40.106 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:09:40.106 INFO:teuthology.orchestra.run.smithi033.stdout: "auth_dump": [ 2024-04-25T01:09:40.106 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.a", 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClm/1anCBAAAd5rluwqED/HsqyQXnS4bw==", 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.b", 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmceVqFxAADWwU51ebqTMdk9oRKfRyqw==", 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.107 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.c", 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmadkeERAAWiEIDRFoI3UFC0gmjTUCzg==", 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2024-04-25T01:09:40.108 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mds.d", 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmzuw8HRAArbWjoWoWc9QnmClOslRhyA==", 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.109 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.0", 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDXrClmO46RKxAA9f2l5EWxqEUjAeg2/dZ7BQ==", 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.110 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.1", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDarClmjBi6MRAA1HxP2UL7163J1a6EatsCrQ==", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.2", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDdrClmucXKOhAAaenJnSHXygXuJw9D3uiSIg==", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.111 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.3", 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDhrClmHO+kBRAAi2Mtq+PLtDWRZtv+Jb+HnA==", 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.4", 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDnrClmORmeAhAAf5kNhiCpkd0oJAVYReRnfw==", 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.5", 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDqrClm0zMpBxAAx9DWUbvSD29vy0aj0KbphQ==", 2024-04-25T01:09:40.113 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.6", 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDtrClm26g+DRAAgR+9igOsmcAO4/cYVfW40g==", 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.114 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "osd.7", 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDwrClmyBEIEhAAZT4pGOBNnrffGisaecuD0g==", 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow profile osd", 2024-04-25T01:09:40.116 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile osd", 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.0", 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmzPTbIhAAvzORiMqrE/Y+uTLvKO5BMw==", 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2024-04-25T01:09:40.117 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.1", 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmegPyKhAAHLA/5dqn09dPyHPr/X6z3w==", 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.118 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.2", 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClme7kQKBAAtTHHa+QxjysQ5Gzb8+DNLw==", 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.3", 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmzPDQLxAApaIke8y8GYt0E2+ZODUjiA==", 2024-04-25T01:09:40.119 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow r", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow rw", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow rwx" 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.admin", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDTrClmXiTaLBAAL67Nc3L8d18iT5X9HoT7yA==", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": "allow *", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow *", 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.120 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mds", 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD2rClmuQzPGRAADWnXUqk2+qDCQvoAGxF3uQ==", 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD2rClm1SfPGRAA5TDg9xJcDv0KdnX5IFNopw==", 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T01:09:40.121 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-osd", 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD2rClm9z3PGRAAkwZLIaXrm5eo0u76uz5rGQ==", 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD2rClmelPPGRAAOR46wcxTrjH6scfaf33xdQ==", 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T01:09:40.122 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD2rClm1WnPGRAAhWA+ErcZah3HNKvjhB24CQ==", 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQD2rClmvIDPGRAAHSwAt2vQpke0oxtC1eumJQ==", 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T01:09:40.123 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.x", 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmoJwYBRAANlFwhdQApqMEg1FRrFyLGA==", 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "entity": "mgr.y", 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "key": "AQDUrClmG2U4AhAAJWFzFjoGGiuamFr4u+FWeA==", 2024-04-25T01:09:40.124 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": { 2024-04-25T01:09:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": "allow *", 2024-04-25T01:09:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": "allow profile mgr", 2024-04-25T01:09:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": "allow *" 2024-04-25T01:09:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:40.125 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-25T01:09:40.125 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcc980678c0 msgr2=0x7fcc98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcc980678c0 0x7fcc98069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0b2000 tx=0x7fcc9c0aa070 comp rx=0 tx=0).stop 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 msgr2=0x7fccac09f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac09f560 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcca000c7c0 tx=0x7fcca000cc80 comp rx=0 tx=0).stop 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 shutdown_connections 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcc980678c0 0x7fcc98069d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fccac0d2200 0x7fccac1f3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccac0d1e20 0x7fccac09f560 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 --2- 172.21.15.33:0/273678347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccac0b1010 0x7fccac1460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 >> 172.21.15.33:0/273678347 conn(0x7fccac09a9d0 msgr2=0x7fccac0aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:40.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 shutdown_connections 2024-04-25T01:09:40.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.116+0000 7fccb30c1640 1 -- 172.21.15.33:0/273678347 wait complete. 2024-04-25T01:09:40.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T01:09:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a035d3640 1 Processor -- start 2024-04-25T01:09:40.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a035d3640 1 -- start start 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a035d3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 0x7f79fc0ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a035d3640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc06b540 con 0x7f79fc0afc90 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a035d3640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79fc06b820 con 0x7f79fc0d0bb0 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a035d3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc06ccf0 con 0x7f79fc0ad1d0 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a0136f640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f79fc0afc90 0x7f79fc0b0070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39778/0 (socket says 172.21.15.33:39778) 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a0136f640 1 -- 172.21.15.33:0/1632199659 learned_addr learned my addr 172.21.15.33:0/1632199659 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a01b70640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 0x7f79fc0ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711547167 0 0) 0x7f79fc06b540 con 0x7f79fc0afc90 2024-04-25T01:09:40.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79e4003530 con 0x7f79fc0afc90 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3160911326 0 0) 0x7f79e4003530 con 0x7f79fc0afc90 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 msgr2=0x7f79fc0ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 0x7f79fc0ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 >> v1:172.21.15.138:6789/0 conn(0x7f79fc0d0bb0 legacy=0x7f79fc06c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc06bb00 con 0x7f79fc0afc90 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f79ec001a00 con 0x7f79fc0afc90 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 >> v1:172.21.15.33:6789/0 conn(0x7f79fc0afc90 legacy=0x7f79fc0b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e40039f0 0x7f79e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79e4004810 0x7f79e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 0x7f79e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc06ccf0 con 0x7f79e4004810 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79e400bb30 con 0x7f79e40039f0 2024-04-25T01:09:40.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79e400df40 con 0x7f79fc0ad1d0 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a00b6e640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79e4004810 0x7f79e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a0136f640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e40039f0 0x7f79e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.264+0000 7f7a01b70640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 0x7f79e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a00b6e640 1 -- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 msgr2=0x7f79e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a00b6e640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 0x7f79e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a00b6e640 1 -- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e40039f0 msgr2=0x7f79e4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a00b6e640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e40039f0 0x7f79e4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a00b6e640 1 -- 172.21.15.33:0/1632199659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e4003530 con 0x7f79e4004810 2024-04-25T01:09:40.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a01b70640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0ad1d0 0x7f79e4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a00b6e640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79e4004810 0x7f79e4004c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f79f4001780 tx=0x7f79f401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda08b4e37b4e404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f40013a0 con 0x7f79e4004810 2024-04-25T01:09:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f40052f0 con 0x7f79e4004810 2024-04-25T01:09:40.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f79fbfff640 1 -- 172.21.15.33:0/1632199659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4003840 con 0x7f79e4004810 2024-04-25T01:09:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a035d3640 1 -- 172.21.15.33:0/1632199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79e4004810 msgr2=0x7f79e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a035d3640 1 --2- 172.21.15.33:0/1632199659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79e4004810 0x7f79e4004c90 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f79f4001780 tx=0x7f79f401c420 comp rx=0 tx=0).stop 2024-04-25T01:09:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a01b70640 1 -- 172.21.15.33:0/1632199659 reap_dead start 2024-04-25T01:09:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a035d3640 1 -- 172.21.15.33:0/1632199659 shutdown_connections 2024-04-25T01:09:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a035d3640 1 -- 172.21.15.33:0/1632199659 >> 172.21.15.33:0/1632199659 conn(0x7f79fc0b3df0 msgr2=0x7f79fc0b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a035d3640 1 -- 172.21.15.33:0/1632199659 shutdown_connections 2024-04-25T01:09:40.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.268+0000 7f7a035d3640 1 -- 172.21.15.33:0/1632199659 wait complete. 2024-04-25T01:09:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 Processor -- start 2024-04-25T01:09:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 -- start start 2024-04-25T01:09:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e4004810 0x7f79fc0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79fc14de80 0x7f79fc0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0cbe90 0x7f79fc13f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc15a860 con 0x7f79fc14de80 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc15a6e0 con 0x7f79e4004810 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79fc0d0fc0 con 0x7f79fc0cbe90 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79fc14de80 0x7f79fc0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79fc14de80 0x7f79fc0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53630/0 (socket says 172.21.15.33:53630) 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 -- 172.21.15.33:0/1008701092 learned_addr learned my addr 172.21.15.33:0/1008701092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a0136f640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e4004810 0x7f79fc0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 -- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0cbe90 msgr2=0x7f79fc13f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a01b70640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0cbe90 0x7f79fc13f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0cbe90 0x7f79fc13f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 -- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e4004810 msgr2=0x7f79fc0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e4004810 0x7f79fc0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 -- 172.21.15.33:0/1008701092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc152280 con 0x7f79fc14de80 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a01b70640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0cbe90 0x7f79fc13f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a00b6e640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79fc14de80 0x7f79fc0cb950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f79f4005040 tx=0x7f79e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f79f9ffb640 1 -- 172.21.15.33:0/1008701092 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f4007a40 con 0x7f79fc14de80 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f79f9ffb640 1 -- 172.21.15.33:0/1008701092 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79f40bb070 con 0x7f79fc14de80 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc152500 con 0x7f79fc14de80 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f79f9ffb640 1 -- 172.21.15.33:0/1008701092 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79f40013a0 con 0x7f79fc14de80 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc152ac0 con 0x7f79fc14de80 2024-04-25T01:09:40.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc069040 con 0x7f79fc14de80 2024-04-25T01:09:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f79f9ffb640 1 -- 172.21.15.33:0/1008701092 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f79f40b4040 con 0x7f79fc14de80 2024-04-25T01:09:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f79f9ffb640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79e80678f0 0x7f79e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.272+0000 7f79f9ffb640 1 -- 172.21.15.33:0/1008701092 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f79f411faa0 con 0x7f79fc14de80 2024-04-25T01:09:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.276+0000 7f7a0136f640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79e80678f0 0x7f79e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.276+0000 7f7a0136f640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79e80678f0 0x7f79e8069db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f79ec002240 tx=0x7f79ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:40.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.280+0000 7f79f9ffb640 1 -- 172.21.15.33:0/1008701092 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f79fc152280 con 0x7f79fc14de80 2024-04-25T01:09:40.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.496+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f79fc0cffb0 con 0x7f79fc14de80 2024-04-25T01:09:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.500+0000 7f79f9ffb640 1 -- 172.21.15.33:0/1008701092 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f79f4016130 con 0x7f79fc14de80 2024-04-25T01:09:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79e80678f0 msgr2=0x7f79e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79e80678f0 0x7f79e8069db0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f79ec002240 tx=0x7f79ec009040 comp rx=0 tx=0).stop 2024-04-25T01:09:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79fc14de80 msgr2=0x7f79fc0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79fc14de80 0x7f79fc0cb950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f79f4005040 tx=0x7f79e4005840 comp rx=0 tx=0).stop 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 shutdown_connections 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79e80678f0 0x7f79e8069db0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f79fc0cbe90 0x7f79fc13f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79fc14de80 0x7f79fc0cb950 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 --2- 172.21.15.33:0/1008701092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79e4004810 0x7f79fc0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 >> 172.21.15.33:0/1008701092 conn(0x7f79fc0b3df0 msgr2=0x7f79fc153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 shutdown_connections 2024-04-25T01:09:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.504+0000 7f7a035d3640 1 -- 172.21.15.33:0/1008701092 wait complete. 2024-04-25T01:09:40.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T01:09:40.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 Processor -- start 2024-04-25T01:09:40.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 -- start start 2024-04-25T01:09:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e41444f0 0x7f94e40b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f94e415e660 0x7f94e4071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94e4072370 0x7f94e406dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f94e40ce0c0 con 0x7f94e4072370 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f94e41448d0 con 0x7f94e415e660 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ec7a2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f94e40b5b60 con 0x7f94e41444f0 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94e4072370 0x7f94e406dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f94e4072370 0x7f94e406dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53638/0 (socket says 172.21.15.33:53638) 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 -- 172.21.15.33:0/3433649684 learned_addr learned my addr 172.21.15.33:0/3433649684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94e9d3d640 1 --2- 172.21.15.33:0/3433649684 >> v2:172.21.15.138:3300/0 conn(0x7f94e415e660 0x7f94e4071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ea53e640 1 --2- 172.21.15.33:0/3433649684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e41444f0 0x7f94e40b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 -- 172.21.15.33:0/3433649684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e41444f0 msgr2=0x7f94e40b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 --2- 172.21.15.33:0/3433649684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e41444f0 0x7f94e40b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 -- 172.21.15.33:0/3433649684 >> v2:172.21.15.138:3300/0 conn(0x7f94e415e660 msgr2=0x7f94e4071e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 --2- 172.21.15.33:0/3433649684 >> v2:172.21.15.138:3300/0 conn(0x7f94e415e660 0x7f94e4071e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.640+0000 7f94ead3f640 1 -- 172.21.15.33:0/3433649684 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e415ec20 con 0x7f94e4072370 2024-04-25T01:09:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ead3f640 1 --2- 172.21.15.33:0/3433649684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4072370 0x7f94e406dab0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f94e406eb70 tx=0x7f94dc01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e923fc10cfa406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94e953c640 1 -- 172.21.15.33:0/3433649684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0a9430 con 0x7f94e4072370 2024-04-25T01:09:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94e953c640 1 -- 172.21.15.33:0/3433649684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94dc0ae070 con 0x7f94e4072370 2024-04-25T01:09:40.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94e953c640 1 -- 172.21.15.33:0/3433649684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0a97f0 con 0x7f94e4072370 2024-04-25T01:09:40.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 -- 172.21.15.33:0/3433649684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4072370 msgr2=0x7f94e406dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/3433649684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4072370 0x7f94e406dab0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f94e406eb70 tx=0x7f94dc01c6a0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 -- 172.21.15.33:0/3433649684 shutdown_connections 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/3433649684 >> v2:172.21.15.138:3300/0 conn(0x7f94e415e660 0x7f94e4071e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/3433649684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e4072370 0x7f94e406dab0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/3433649684 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e41444f0 0x7f94e40b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 -- 172.21.15.33:0/3433649684 >> 172.21.15.33:0/3433649684 conn(0x7f94e40b1810 msgr2=0x7f94e40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 -- 172.21.15.33:0/3433649684 shutdown_connections 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 -- 172.21.15.33:0/3433649684 wait complete. 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 Processor -- start 2024-04-25T01:09:40.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.644+0000 7f94ec7a2640 1 -- start start 2024-04-25T01:09:40.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e4072370 0x7f94e415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e41444f0 0x7f94e415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e415e660 0x7f94e41352d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94e40b37a0 con 0x7f94e41444f0 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f94e40b3620 con 0x7f94e415e660 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f94e4070d20 con 0x7f94e4072370 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e41444f0 0x7f94e415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e41444f0 0x7f94e415a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53640/0 (socket says 172.21.15.33:53640) 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 -- 172.21.15.33:0/1586671411 learned_addr learned my addr 172.21.15.33:0/1586671411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ea53e640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e4072370 0x7f94e415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ead3f640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e415e660 0x7f94e41352d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 -- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e4072370 msgr2=0x7f94e415b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e4072370 0x7f94e415b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 -- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e415e660 msgr2=0x7f94e41352d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e415e660 0x7f94e41352d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 -- 172.21.15.33:0/1586671411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e41647e0 con 0x7f94e41444f0 2024-04-25T01:09:40.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94e9d3d640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e41444f0 0x7f94e415a090 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f94d400d620 tx=0x7f94d400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:40.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94daffd640 1 -- 172.21.15.33:0/1586671411 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d400b620 con 0x7f94e41444f0 2024-04-25T01:09:40.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94daffd640 1 -- 172.21.15.33:0/1586671411 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d4019070 con 0x7f94e41444f0 2024-04-25T01:09:40.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e4164ac0 con 0x7f94e41444f0 2024-04-25T01:09:40.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94daffd640 1 -- 172.21.15.33:0/1586671411 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d400b9e0 con 0x7f94e41444f0 2024-04-25T01:09:40.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e4165080 con 0x7f94e41444f0 2024-04-25T01:09:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e4069040 con 0x7f94e41444f0 2024-04-25T01:09:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94daffd640 1 -- 172.21.15.33:0/1586671411 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f94d400bb80 con 0x7f94e41444f0 2024-04-25T01:09:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94daffd640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f94c80678c0 0x7f94c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94daffd640 1 -- 172.21.15.33:0/1586671411 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f94d4085f10 con 0x7f94e41444f0 2024-04-25T01:09:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.648+0000 7f94ea53e640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f94c80678c0 0x7f94c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:40.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.652+0000 7f94ea53e640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f94c80678c0 0x7f94c8069d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b9d0 tx=0x7f94e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:40.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.652+0000 7f94daffd640 1 -- 172.21.15.33:0/1586671411 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f94d4056b80 con 0x7f94e41444f0 2024-04-25T01:09:40.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.876+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f94e40b2f20 con 0x7f94e41444f0 2024-04-25T01:09:40.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.880+0000 7f94daffd640 1 -- 172.21.15.33:0/1586671411 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f94d4056480 con 0x7f94e41444f0 2024-04-25T01:09:40.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f94c80678c0 msgr2=0x7f94c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f94c80678c0 0x7f94c8069d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b9d0 tx=0x7f94e0008040 comp rx=0 tx=0).stop 2024-04-25T01:09:40.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e41444f0 msgr2=0x7f94e415a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e41444f0 0x7f94e415a090 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f94d400d620 tx=0x7f94d400dae0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 shutdown_connections 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f94c80678c0 0x7f94c8069d80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94e415e660 0x7f94e41352d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94e41444f0 0x7f94e415a090 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 --2- 172.21.15.33:0/1586671411 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f94e4072370 0x7f94e415b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 >> 172.21.15.33:0/1586671411 conn(0x7f94e40b1810 msgr2=0x7f94e413bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 shutdown_connections 2024-04-25T01:09:40.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:40.884+0000 7f94ec7a2640 1 -- 172.21.15.33:0/1586671411 wait complete. 2024-04-25T01:09:40.892 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T01:09:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd461ee0640 1 Processor -- start 2024-04-25T01:09:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd461ee0640 1 -- start start 2024-04-25T01:09:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd461ee0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd45c0d0310 0x7fd45c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd461ee0640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd45c0d26b0 0x7fd45c0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd461ee0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd45c0b0750 con 0x7fd45c15f6d0 2024-04-25T01:09:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd461ee0640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fd45c0b0c30 con 0x7fd45c0d26b0 2024-04-25T01:09:41.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd461ee0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd45c0b26e0 con 0x7fd45c0d0310 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd45c0d26b0 0x7fd45c0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45bfff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd45c0d26b0 0x7fd45c0b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53648/0 (socket says 172.21.15.33:53648) 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45bfff640 1 -- 172.21.15.33:0/2534537283 learned_addr learned my addr 172.21.15.33:0/2534537283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45affd640 1 --2- 172.21.15.33:0/2534537283 >> v2:172.21.15.138:3300/0 conn(0x7fd45c0d0310 0x7fd45c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45bfff640 1 -- 172.21.15.33:0/2534537283 >> v2:172.21.15.138:3300/0 conn(0x7fd45c0d0310 msgr2=0x7fd45c0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45bfff640 1 --2- 172.21.15.33:0/2534537283 >> v2:172.21.15.138:3300/0 conn(0x7fd45c0d0310 0x7fd45c0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45bfff640 1 -- 172.21.15.33:0/2534537283 >> v1:172.21.15.138:6789/0 conn(0x7fd45c15f6d0 legacy=0x7fd45c0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:41.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.020+0000 7fd45bfff640 1 -- 172.21.15.33:0/2534537283 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c0b29c0 con 0x7fd45c0d26b0 2024-04-25T01:09:41.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd45bfff640 1 --2- 172.21.15.33:0/2534537283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c0b18e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd45c068d20 tx=0x7fd45401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6c71295344caa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd45a7fc640 1 -- 172.21.15.33:0/2534537283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540a5670 con 0x7fd45c0d26b0 2024-04-25T01:09:41.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd45a7fc640 1 -- 172.21.15.33:0/2534537283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4540a5810 con 0x7fd45c0d26b0 2024-04-25T01:09:41.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd45a7fc640 1 -- 172.21.15.33:0/2534537283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4540a5be0 con 0x7fd45c0d26b0 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 -- 172.21.15.33:0/2534537283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 msgr2=0x7fd45c0b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 --2- 172.21.15.33:0/2534537283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c0b18e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd45c068d20 tx=0x7fd45401c9b0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 -- 172.21.15.33:0/2534537283 shutdown_connections 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 --2- 172.21.15.33:0/2534537283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c0b18e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 --2- 172.21.15.33:0/2534537283 >> v2:172.21.15.138:3300/0 conn(0x7fd45c0d0310 0x7fd45c0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 -- 172.21.15.33:0/2534537283 >> 172.21.15.33:0/2534537283 conn(0x7fd45c0d1b90 msgr2=0x7fd45c0d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 -- 172.21.15.33:0/2534537283 shutdown_connections 2024-04-25T01:09:41.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.024+0000 7fd461ee0640 1 -- 172.21.15.33:0/2534537283 wait complete. 2024-04-25T01:09:41.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 Processor -- start 2024-04-25T01:09:41.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 -- start start 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd45c0d0310 0x7fd45c15d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd45c15f6d0 0x7fd45c0c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c169060 con 0x7fd45c0d26b0 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd45c168ee0 con 0x7fd45c0d0310 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd45c1691e0 con 0x7fd45c15f6d0 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53650/0 (socket says 172.21.15.33:53650) 2024-04-25T01:09:41.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 -- 172.21.15.33:0/1772379401 learned_addr learned my addr 172.21.15.33:0/1772379401 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 -- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd45c15f6d0 msgr2=0x7fd45c0c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45bfff640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd45c15f6d0 0x7fd45c0c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45b7fe640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd45c0d0310 0x7fd45c15d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd45c15f6d0 0x7fd45c0c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 -- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd45c0d0310 msgr2=0x7fd45c15d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd45c0d0310 0x7fd45c15d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 -- 172.21.15.33:0/1772379401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd45c15eb80 con 0x7fd45c0d26b0 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45bfff640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd45c15f6d0 0x7fd45c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45b7fe640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd45c0d0310 0x7fd45c15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd45affd640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c15e640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd45000d800 tx=0x7fd45000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd443fff640 1 -- 172.21.15.33:0/1772379401 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45000b6c0 con 0x7fd45c0d26b0 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd443fff640 1 -- 172.21.15.33:0/1772379401 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd450019070 con 0x7fd45c0d26b0 2024-04-25T01:09:41.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd45c0c6be0 con 0x7fd45c0d26b0 2024-04-25T01:09:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd443fff640 1 -- 172.21.15.33:0/1772379401 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45000ba60 con 0x7fd45c0d26b0 2024-04-25T01:09:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd45c0c71a0 con 0x7fd45c0d26b0 2024-04-25T01:09:41.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd45c069040 con 0x7fd45c0d26b0 2024-04-25T01:09:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd443fff640 1 -- 172.21.15.33:0/1772379401 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd45000bc00 con 0x7fd45c0d26b0 2024-04-25T01:09:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.028+0000 7fd443fff640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd4480678c0 0x7fd448069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.032+0000 7fd443fff640 1 -- 172.21.15.33:0/1772379401 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fd450085fe0 con 0x7fd45c0d26b0 2024-04-25T01:09:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.032+0000 7fd45b7fe640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd4480678c0 0x7fd448069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.032+0000 7fd45b7fe640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd4480678c0 0x7fd448069d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00a850 tx=0x7fd44c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.036+0000 7fd443fff640 1 -- 172.21.15.33:0/1772379401 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd450056c40 con 0x7fd45c0d26b0 2024-04-25T01:09:41.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.256+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fd45c15ee20 con 0x7fd45c0d26b0 2024-04-25T01:09:41.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.260+0000 7fd443fff640 1 -- 172.21.15.33:0/1772379401 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fd450056540 con 0x7fd45c0d26b0 2024-04-25T01:09:41.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd4480678c0 msgr2=0x7fd448069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd4480678c0 0x7fd448069d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00a850 tx=0x7fd44c008040 comp rx=0 tx=0).stop 2024-04-25T01:09:41.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 msgr2=0x7fd45c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c15e640 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd45000d800 tx=0x7fd45000dcc0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 shutdown_connections 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd4480678c0 0x7fd448069d80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd45c15f6d0 0x7fd45c0c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd45c0d26b0 0x7fd45c15e640 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 --2- 172.21.15.33:0/1772379401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd45c0d0310 0x7fd45c15d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 >> 172.21.15.33:0/1772379401 conn(0x7fd45c0d1b90 msgr2=0x7fd45c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 shutdown_connections 2024-04-25T01:09:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.264+0000 7fd461ee0640 1 -- 172.21.15.33:0/1772379401 wait complete. 2024-04-25T01:09:41.274 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T01:09:41.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa10598c640 1 Processor -- start 2024-04-25T01:09:41.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa10598c640 1 -- start start 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa10598c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fa1000ceb30 0x7fa10006fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa10598c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa1000cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa10598c640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1000cfe60 con 0x7fa1000cd9a0 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa10598c640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fa1000d0100 con 0x7fa1000ceb30 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa10598c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1000d03e0 con 0x7fa1000cdd80 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0feffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa1000cd9a0 0x7fa1000ce420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39794/0 (socket says 172.21.15.33:39794) 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0feffd640 1 -- 172.21.15.33:0/3892092876 learned_addr learned my addr 172.21.15.33:0/3892092876 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0fe7fc640 1 --2- 172.21.15.33:0/3892092876 >> v2:172.21.15.33:3300/0 conn(0x7fa1000ceb30 0x7fa10006fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0ff7fe640 1 --2- 172.21.15.33:0/3892092876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa1000cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0fdffb640 1 -- 172.21.15.33:0/3892092876 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104646729 0 0) 0x7fa1000cfe60 con 0x7fa1000cd9a0 2024-04-25T01:09:41.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0fdffb640 1 -- 172.21.15.33:0/3892092876 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0ec003530 con 0x7fa1000cd9a0 2024-04-25T01:09:41.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0fe7fc640 1 -- 172.21.15.33:0/3892092876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 msgr2=0x7fa1000cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:41.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0fe7fc640 1 --2- 172.21.15.33:0/3892092876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa1000cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0fe7fc640 1 -- 172.21.15.33:0/3892092876 >> v1:172.21.15.33:6789/0 conn(0x7fa1000cd9a0 legacy=0x7fa1000ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0fe7fc640 1 -- 172.21.15.33:0/3892092876 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1000d06c0 con 0x7fa1000ceb30 2024-04-25T01:09:41.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.396+0000 7fa0ff7fe640 1 --2- 172.21.15.33:0/3892092876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa1000cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:41.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa0fe7fc640 1 --2- 172.21.15.33:0/3892092876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000ceb30 0x7fa10006fdc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80097c0 tx=0x7fa0e801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dac3145112418e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa0fdffb640 1 -- 172.21.15.33:0/3892092876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80aa5f0 con 0x7fa1000ceb30 2024-04-25T01:09:41.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa0fdffb640 1 -- 172.21.15.33:0/3892092876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e80af070 con 0x7fa1000ceb30 2024-04-25T01:09:41.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa0fdffb640 1 -- 172.21.15.33:0/3892092876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e80aa9b0 con 0x7fa1000ceb30 2024-04-25T01:09:41.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 -- 172.21.15.33:0/3892092876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000ceb30 msgr2=0x7fa10006fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 --2- 172.21.15.33:0/3892092876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000ceb30 0x7fa10006fdc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80097c0 tx=0x7fa0e801c470 comp rx=0 tx=0).stop 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 -- 172.21.15.33:0/3892092876 shutdown_connections 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 --2- 172.21.15.33:0/3892092876 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa1000cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 --2- 172.21.15.33:0/3892092876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000ceb30 0x7fa10006fdc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 -- 172.21.15.33:0/3892092876 >> 172.21.15.33:0/3892092876 conn(0x7fa1000d1ba0 msgr2=0x7fa1000d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 -- 172.21.15.33:0/3892092876 shutdown_connections 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 -- 172.21.15.33:0/3892092876 wait complete. 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.400+0000 7fa10598c640 1 Processor -- start 2024-04-25T01:09:41.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 -- start start 2024-04-25T01:09:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000cd9a0 0x7fa1001506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa10014e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1000ceb30 0x7fa100156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa10016b140 con 0x7fa1000cd9a0 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1000d10d0 con 0x7fa1000ceb30 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa10016b2c0 con 0x7fa1000cdd80 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000cd9a0 0x7fa1001506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000cd9a0 0x7fa1001506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53672/0 (socket says 172.21.15.33:53672) 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 -- 172.21.15.33:0/582644457 learned_addr learned my addr 172.21.15.33:0/582644457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 -- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 msgr2=0x7fa10014e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa10014e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 -- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1000ceb30 msgr2=0x7fa100156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1000ceb30 0x7fa100156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 -- 172.21.15.33:0/582644457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10014f040 con 0x7fa1000cd9a0 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0feffd640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000cd9a0 0x7fa1001506e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800cb40 tx=0x7fa0f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0f7fff640 1 -- 172.21.15.33:0/582644457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8017070 con 0x7fa1000cd9a0 2024-04-25T01:09:41.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0f7fff640 1 -- 172.21.15.33:0/582644457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f800aca0 con 0x7fa1000cd9a0 2024-04-25T01:09:41.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0f7fff640 1 -- 172.21.15.33:0/582644457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f8012620 con 0x7fa1000cd9a0 2024-04-25T01:09:41.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa100152f60 con 0x7fa1000cd9a0 2024-04-25T01:09:41.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa100153520 con 0x7fa1000cd9a0 2024-04-25T01:09:41.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa100069040 con 0x7fa1000cd9a0 2024-04-25T01:09:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0f7fff640 1 -- 172.21.15.33:0/582644457 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0f801a070 con 0x7fa1000cd9a0 2024-04-25T01:09:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0f7fff640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa0e00678c0 0x7fa0e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0f7fff640 1 -- 172.21.15.33:0/582644457 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fa0f80859a0 con 0x7fa1000cd9a0 2024-04-25T01:09:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.404+0000 7fa0fe7fc640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa0e00678c0 0x7fa0e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.408+0000 7fa0fe7fc640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa0e00678c0 0x7fa0e0069d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ae000 tx=0x7fa0e8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.408+0000 7fa0f7fff640 1 -- 172.21.15.33:0/582644457 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa0f8056610 con 0x7fa1000cd9a0 2024-04-25T01:09:41.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.628+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fa1000b7b20 con 0x7fa1000cd9a0 2024-04-25T01:09:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.632+0000 7fa0f7fff640 1 -- 172.21.15.33:0/582644457 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7fa0f8055f10 con 0x7fa1000cd9a0 2024-04-25T01:09:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa0e00678c0 msgr2=0x7fa0e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa0e00678c0 0x7fa0e0069d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ae000 tx=0x7fa0e8004740 comp rx=0 tx=0).stop 2024-04-25T01:09:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000cd9a0 msgr2=0x7fa1001506e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000cd9a0 0x7fa1001506e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa0f800cb40 tx=0x7fa0f800a060 comp rx=0 tx=0).stop 2024-04-25T01:09:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 shutdown_connections 2024-04-25T01:09:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa0e00678c0 0x7fa0e0069d80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1000ceb30 0x7fa100156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1000cdd80 0x7fa10014e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 --2- 172.21.15.33:0/582644457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1000cd9a0 0x7fa1001506e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 >> 172.21.15.33:0/582644457 conn(0x7fa1000d1ba0 msgr2=0x7fa1000aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 shutdown_connections 2024-04-25T01:09:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.636+0000 7fa10598c640 1 -- 172.21.15.33:0/582644457 wait complete. 2024-04-25T01:09:41.643 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T01:09:41.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b7ff5640 1 Processor -- start 2024-04-25T01:09:41.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b7ff5640 1 -- start start 2024-04-25T01:09:41.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b7ff5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b00b4c50 0x7f38b015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b7ff5640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f38b00b5680 0x7f38b00b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b7ff5640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38b006f020 con 0x7f38b01563d0 2024-04-25T01:09:41.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b7ff5640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f38b00b6030 con 0x7f38b00b5680 2024-04-25T01:09:41.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b7ff5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f38b006f2a0 con 0x7f38b00b4c50 2024-04-25T01:09:41.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f38b01563d0 0x7f38b015cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39806/0 (socket says 172.21.15.33:39806) 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 -- 172.21.15.33:0/2819884247 learned_addr learned my addr 172.21.15.33:0/2819884247 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b6592640 1 --2- 172.21.15.33:0/2819884247 >> v2:172.21.15.138:3300/0 conn(0x7f38b00b5680 0x7f38b00b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5590640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b00b4c50 0x7f38b015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1234738824 0 0) 0x7f38ac001120 con 0x7f38b01563d0 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f389c003530 con 0x7f38b01563d0 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3098924767 0 0) 0x7f389c003530 con 0x7f38b01563d0 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b00b4c50 msgr2=0x7f38b015d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b00b4c50 0x7f38b015d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 >> v2:172.21.15.138:3300/0 conn(0x7f38b00b5680 msgr2=0x7f38b00b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 --2- 172.21.15.33:0/2819884247 >> v2:172.21.15.138:3300/0 conn(0x7f38b00b5680 0x7f38b00b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b006f580 con 0x7f38b01563d0 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f38ac001850 con 0x7f38b01563d0 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 >> v1:172.21.15.33:6789/0 conn(0x7f38b01563d0 legacy=0x7f38b015cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0039c0 0x7f389c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f389c004810 0x7f389c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b00b4c50 0x7f389c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38b00b6030 con 0x7f389c0039c0 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38b006f2a0 con 0x7f389c004810 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f389c00ba20 con 0x7f38b00b4c50 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0039c0 0x7f389c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5590640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f389c004810 0x7f389c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 -- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b00b4c50 msgr2=0x7f389c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38b00b4c50 0x7f389c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 -- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f389c004810 msgr2=0x7f389c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f389c004810 0x7f389c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.780+0000 7f38b5d91640 1 -- 172.21.15.33:0/2819884247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c003530 con 0x7f389c0039c0 2024-04-25T01:09:41.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0039c0 0x7f389c003de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c350 tx=0x7f38ac01cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f23b5a8ce4821e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac007e40 con 0x7f389c0039c0 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38ac0a9020 con 0x7f389c0039c0 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b4d8f640 1 -- 172.21.15.33:0/2819884247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac005b00 con 0x7f389c0039c0 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2819884247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0039c0 msgr2=0x7f389c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 --2- 172.21.15.33:0/2819884247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f389c0039c0 0x7f389c003de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f38ac00c350 tx=0x7f38ac01cbc0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b6592640 1 -- 172.21.15.33:0/2819884247 reap_dead start 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2819884247 shutdown_connections 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2819884247 >> 172.21.15.33:0/2819884247 conn(0x7f38b00734c0 msgr2=0x7f38b009a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2819884247 shutdown_connections 2024-04-25T01:09:41.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2819884247 wait complete. 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 Processor -- start 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.784+0000 7f38b7ff5640 1 -- start start 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f389c0039c0 0x7f38b01496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0154ff0 0x7f38b0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b0158ba0 0x7f38b0159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0070ee0 con 0x7f38b0154ff0 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38b0070d60 con 0x7f38b0158ba0 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f38b0071060 con 0x7f389c0039c0 2024-04-25T01:09:41.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0154ff0 0x7f38b0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0154ff0 0x7f38b0155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53694/0 (socket says 172.21.15.33:53694) 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 -- 172.21.15.33:0/2935426269 learned_addr learned my addr 172.21.15.33:0/2935426269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f389c0039c0 0x7f38b01496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b6592640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b0158ba0 0x7f38b0159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 -- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f389c0039c0 msgr2=0x7f38b01496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f389c0039c0 0x7f38b01496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 -- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b0158ba0 msgr2=0x7f38b0159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b0158ba0 0x7f38b0159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 -- 172.21.15.33:0/2935426269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38b01f64c0 con 0x7f38b0154ff0 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f389c0039c0 0x7f38b01496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b6592640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b0158ba0 0x7f38b0159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5590640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0154ff0 0x7f38b0155450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f38a0010890 tx=0x7f38a0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38a67fc640 1 -- 172.21.15.33:0/2935426269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a000f040 con 0x7f38b0154ff0 2024-04-25T01:09:41.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38a67fc640 1 -- 172.21.15.33:0/2935426269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a000eb60 con 0x7f38b0154ff0 2024-04-25T01:09:41.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38b01f67a0 con 0x7f38b0154ff0 2024-04-25T01:09:41.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38a67fc640 1 -- 172.21.15.33:0/2935426269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a001c400 con 0x7f38b0154ff0 2024-04-25T01:09:41.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38b01f6d60 con 0x7f38b0154ff0 2024-04-25T01:09:41.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38b0069040 con 0x7f38b0154ff0 2024-04-25T01:09:41.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38a67fc640 1 -- 172.21.15.33:0/2935426269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38a001c5a0 con 0x7f38b0154ff0 2024-04-25T01:09:41.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38a67fc640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38940678c0 0x7f3894069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:41.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38a67fc640 1 -- 172.21.15.33:0/2935426269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f38a0086180 con 0x7f38b0154ff0 2024-04-25T01:09:41.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.788+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38940678c0 0x7f3894069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:41.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.792+0000 7f38b5d91640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38940678c0 0x7f3894069d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0aa000 tx=0x7f38b0067af0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:41.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:41.792+0000 7f38a67fc640 1 -- 172.21.15.33:0/2935426269 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f38a0018030 con 0x7f38b0154ff0 2024-04-25T01:09:42.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.016+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f38b00b4a30 con 0x7f38b0154ff0 2024-04-25T01:09:42.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38a67fc640 1 -- 172.21.15.33:0/2935426269 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f38a0056e80 con 0x7f38b0154ff0 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38940678c0 msgr2=0x7f3894069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38940678c0 0x7f3894069d80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f38ac0aa000 tx=0x7f38b0067af0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0154ff0 msgr2=0x7f38b0155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0154ff0 0x7f38b0155450 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f38a0010890 tx=0x7f38a0010d50 comp rx=0 tx=0).stop 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 shutdown_connections 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38940678c0 0x7f3894069d80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38b0158ba0 0x7f38b0159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38b0154ff0 0x7f38b0155450 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 --2- 172.21.15.33:0/2935426269 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f389c0039c0 0x7f38b01496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 >> 172.21.15.33:0/2935426269 conn(0x7f38b00734c0 msgr2=0x7f38b0162300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 shutdown_connections 2024-04-25T01:09:42.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.020+0000 7f38b7ff5640 1 -- 172.21.15.33:0/2935426269 wait complete. 2024-04-25T01:09:42.031 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T01:09:42.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8307dc4640 1 Processor -- start 2024-04-25T01:09:42.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8307dc4640 1 -- start start 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8307dc4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83001444f0 0x7f83000b5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8307dc4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f830015e660 0x7f8300071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8307dc4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83001448d0 con 0x7f8300072370 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8307dc4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f83000ce0c0 con 0x7f830015e660 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8307dc4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83000b5b60 con 0x7f83001444f0 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f830535f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f830015e660 0x7f8300071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8306361640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f8300072370 0x7f830006dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39812/0 (socket says 172.21.15.33:39812) 2024-04-25T01:09:42.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8306361640 1 -- 172.21.15.33:0/2051714371 learned_addr learned my addr 172.21.15.33:0/2051714371 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8305b60640 1 --2- 172.21.15.33:0/2051714371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83001444f0 0x7f83000b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f830535f640 1 -- 172.21.15.33:0/2051714371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83001444f0 msgr2=0x7f83000b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f830535f640 1 --2- 172.21.15.33:0/2051714371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83001444f0 0x7f83000b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f830535f640 1 -- 172.21.15.33:0/2051714371 >> v1:172.21.15.33:6789/0 conn(0x7f8300072370 legacy=0x7f830006dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f830535f640 1 -- 172.21.15.33:0/2051714371 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830015ec20 con 0x7f830015e660 2024-04-25T01:09:42.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.164+0000 7f8304b5e640 1 -- 172.21.15.33:0/2051714371 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1643094166 0 0) 0x7f82f80014a0 con 0x7f8300072370 2024-04-25T01:09:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f830535f640 1 --2- 172.21.15.33:0/2051714371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015e660 0x7f8300071e30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097c0 tx=0x7f82f401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf420226b2c276 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8304b5e640 1 -- 172.21.15.33:0/2051714371 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f40b0070 con 0x7f830015e660 2024-04-25T01:09:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8304b5e640 1 -- 172.21.15.33:0/2051714371 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f40ab600 con 0x7f830015e660 2024-04-25T01:09:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8304b5e640 1 -- 172.21.15.33:0/2051714371 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f40ab9d0 con 0x7f830015e660 2024-04-25T01:09:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 -- 172.21.15.33:0/2051714371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015e660 msgr2=0x7f8300071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2051714371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015e660 0x7f8300071e30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097c0 tx=0x7f82f401c410 comp rx=0 tx=0).stop 2024-04-25T01:09:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 -- 172.21.15.33:0/2051714371 shutdown_connections 2024-04-25T01:09:42.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2051714371 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f830015e660 0x7f8300071e30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2051714371 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f83001444f0 0x7f83000b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 -- 172.21.15.33:0/2051714371 >> 172.21.15.33:0/2051714371 conn(0x7f83000b1810 msgr2=0x7f83000b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 -- 172.21.15.33:0/2051714371 shutdown_connections 2024-04-25T01:09:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 -- 172.21.15.33:0/2051714371 wait complete. 2024-04-25T01:09:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 Processor -- start 2024-04-25T01:09:42.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.168+0000 7f8307dc4640 1 -- start start 2024-04-25T01:09:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8300072370 0x7f830015b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83001444f0 0x7f8300156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830015e660 0x7f83000b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f83000b3620 con 0x7f83001444f0 2024-04-25T01:09:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83000727c0 con 0x7f830015e660 2024-04-25T01:09:42.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f83000b37a0 con 0x7f8300072370 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83001444f0 0x7f8300156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83001444f0 0x7f8300156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53726/0 (socket says 172.21.15.33:53726) 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 -- 172.21.15.33:0/2295325155 learned_addr learned my addr 172.21.15.33:0/2295325155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8306361640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830015e660 0x7f83000b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8305b60640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8300072370 0x7f830015b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 -- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8300072370 msgr2=0x7f830015b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8300072370 0x7f830015b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 -- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830015e660 msgr2=0x7f83000b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830015e660 0x7f83000b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 -- 172.21.15.33:0/2295325155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83001fa650 con 0x7f83001444f0 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f830535f640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83001444f0 0x7f8300156a60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097c0 tx=0x7f82f401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f82f27fc640 1 -- 172.21.15.33:0/2295325155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f40b0040 con 0x7f83001444f0 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f82f27fc640 1 -- 172.21.15.33:0/2295325155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82f4004030 con 0x7f83001444f0 2024-04-25T01:09:42.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f82f27fc640 1 -- 172.21.15.33:0/2295325155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4004400 con 0x7f83001444f0 2024-04-25T01:09:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83001fa930 con 0x7f83001444f0 2024-04-25T01:09:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83001570e0 con 0x7f83001444f0 2024-04-25T01:09:42.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8300069040 con 0x7f83001444f0 2024-04-25T01:09:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f82f27fc640 1 -- 172.21.15.33:0/2295325155 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f82f4003620 con 0x7f83001444f0 2024-04-25T01:09:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f82f27fc640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f82e40678c0 0x7f82e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f82f27fc640 1 -- 172.21.15.33:0/2295325155 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f82f41207a0 con 0x7f83001444f0 2024-04-25T01:09:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.172+0000 7f8305b60640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f82e40678c0 0x7f82e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.176+0000 7f8305b60640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f82e40678c0 0x7f82e4069d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009790 tx=0x7f82fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.176+0000 7f82f27fc640 1 -- 172.21.15.33:0/2295325155 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f82f4016090 con 0x7f83001444f0 2024-04-25T01:09:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.396+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f83000b2f20 con 0x7f83001444f0 2024-04-25T01:09:42.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.400+0000 7f82f27fc640 1 -- 172.21.15.33:0/2295325155 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f82f40f1530 con 0x7f83001444f0 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f82e40678c0 msgr2=0x7f82e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f82e40678c0 0x7f82e4069d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009790 tx=0x7f82fc008040 comp rx=0 tx=0).stop 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83001444f0 msgr2=0x7f8300156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83001444f0 0x7f8300156a60 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f82f40097c0 tx=0x7f82f401ac50 comp rx=0 tx=0).stop 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 shutdown_connections 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f82e40678c0 0x7f82e4069d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f830015e660 0x7f83000b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83001444f0 0x7f8300156a60 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 --2- 172.21.15.33:0/2295325155 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8300072370 0x7f830015b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 >> 172.21.15.33:0/2295325155 conn(0x7f83000b1810 msgr2=0x7f83000ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 shutdown_connections 2024-04-25T01:09:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.404+0000 7f8307dc4640 1 -- 172.21.15.33:0/2295325155 wait complete. 2024-04-25T01:09:42.413 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.540+0000 7f97e3b6b640 1 Processor -- start 2024-04-25T01:09:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.540+0000 7f97e3b6b640 1 -- start start 2024-04-25T01:09:42.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e3b6b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 0x7f97dc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e3b6b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f97dc0b5360 0x7f97dc0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e3b6b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97dc15f9a0 con 0x7f97dc145470 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e3b6b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f97dc0b57b0 con 0x7f97dc0b5360 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e3b6b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc15f750 con 0x7f97dc15c160 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1907640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f97dc145470 0x7f97dc0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39818/0 (socket says 172.21.15.33:39818) 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1907640 1 -- 172.21.15.33:0/2257946332 learned_addr learned my addr 172.21.15.33:0/2257946332 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e2108640 1 --2- 172.21.15.33:0/2257946332 >> v2:172.21.15.138:3300/0 conn(0x7f97dc0b5360 0x7f97dc0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1106640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 0x7f97dc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2427466716 0 0) 0x7f97dc15f9a0 con 0x7f97dc145470 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f97c8003530 con 0x7f97dc145470 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4007151836 0 0) 0x7f97c8003530 con 0x7f97dc145470 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 msgr2=0x7f97dc0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 0x7f97dc0b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 >> v2:172.21.15.138:3300/0 conn(0x7f97dc0b5360 msgr2=0x7f97dc0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:42.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 --2- 172.21.15.33:0/2257946332 >> v2:172.21.15.138:3300/0 conn(0x7f97dc0b5360 0x7f97dc0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc15fc80 con 0x7f97dc145470 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f97cc001680 con 0x7f97dc145470 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 >> v1:172.21.15.33:6789/0 conn(0x7f97dc145470 legacy=0x7f97dc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80039c0 0x7f97c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c8004810 0x7f97c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 0x7f97c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0b57b0 con 0x7f97c8004810 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc15f750 con 0x7f97c80039c0 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97c800fde0 con 0x7f97dc15c160 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1106640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c8004810 0x7f97c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1907640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80039c0 0x7f97c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e2108640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 0x7f97c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1106640 1 -- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 msgr2=0x7f97c80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:42.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1106640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc15c160 0x7f97c80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1106640 1 -- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80039c0 msgr2=0x7f97c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1106640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80039c0 0x7f97c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1106640 1 -- 172.21.15.33:0/2257946332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c8003530 con 0x7f97c8004810 2024-04-25T01:09:42.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.544+0000 7f97e1907640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c80039c0 0x7f97c8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c8004810 0x7f97c8004c90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009ae0 tx=0x7f97d801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19a52526ebf18c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80b2070 con 0x7f97c8004810 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d80a5580 con 0x7f97c8004810 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e0905640 1 -- 172.21.15.33:0/2257946332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80096b0 con 0x7f97c8004810 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2257946332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c8004810 msgr2=0x7f97c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 --2- 172.21.15.33:0/2257946332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97c8004810 0x7f97c8004c90 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009ae0 tx=0x7f97d801ca70 comp rx=0 tx=0).stop 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e2108640 1 -- 172.21.15.33:0/2257946332 reap_dead start 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2257946332 shutdown_connections 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2257946332 >> 172.21.15.33:0/2257946332 conn(0x7f97dc0b1810 msgr2=0x7f97dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:42.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2257946332 shutdown_connections 2024-04-25T01:09:42.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2257946332 wait complete. 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 Processor -- start 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- start start 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004810 0x7f97dc13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97dc1417a0 0x7f97dc141c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc0cd3a0 0x7f97dc0cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0cf7f0 con 0x7f97dc1417a0 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc160240 con 0x7f97c8004810 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e3b6b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc145850 con 0x7f97dc0cd3a0 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97dc1417a0 0x7f97dc141c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97dc1417a0 0x7f97dc141c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53748/0 (socket says 172.21.15.33:53748) 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 -- 172.21.15.33:0/2483181581 learned_addr learned my addr 172.21.15.33:0/2483181581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:42.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1907640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004810 0x7f97dc13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e2108640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc0cd3a0 0x7f97dc0cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 -- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc0cd3a0 msgr2=0x7f97dc0cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc0cd3a0 0x7f97dc0cd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 -- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004810 msgr2=0x7f97dc13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004810 0x7f97dc13f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.548+0000 7f97e1106640 1 -- 172.21.15.33:0/2483181581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc0aae10 con 0x7f97dc1417a0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97e1106640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97dc1417a0 0x7f97dc141c20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009ae0 tx=0x7f97c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97d27fc640 1 -- 172.21.15.33:0/2483181581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80bc020 con 0x7f97dc1417a0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97d27fc640 1 -- 172.21.15.33:0/2483181581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d80a5db0 con 0x7f97dc1417a0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc0aa290 con 0x7f97dc1417a0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97d27fc640 1 -- 172.21.15.33:0/2483181581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80b2040 con 0x7f97dc1417a0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc0aa850 con 0x7f97dc1417a0 2024-04-25T01:09:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc069040 con 0x7f97dc1417a0 2024-04-25T01:09:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97d27fc640 1 -- 172.21.15.33:0/2483181581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f97d8004220 con 0x7f97dc1417a0 2024-04-25T01:09:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97d27fc640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f97c00678c0 0x7f97c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97d27fc640 1 -- 172.21.15.33:0/2483181581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f97d811fc40 con 0x7f97dc1417a0 2024-04-25T01:09:42.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97e1907640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f97c00678c0 0x7f97c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.552+0000 7f97e1907640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f97c00678c0 0x7f97c0069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f97cc004880 tx=0x7f97cc001880 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.556+0000 7f97d27fc640 1 -- 172.21.15.33:0/2483181581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f97d80c6020 con 0x7f97dc1417a0 2024-04-25T01:09:42.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.716+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97dc071420 con 0x7f97dc1417a0 2024-04-25T01:09:42.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97d27fc640 1 -- 172.21.15.33:0/2483181581 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13152 (secure 0 0 0) 0x7f97d8016780 con 0x7f97dc1417a0 2024-04-25T01:09:42.712 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:42.712 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:39.737859+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f97c00678c0 msgr2=0x7f97c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f97c00678c0 0x7f97c0069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f97cc004880 tx=0x7f97cc001880 comp rx=0 tx=0).stop 2024-04-25T01:09:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97dc1417a0 msgr2=0x7f97dc141c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97dc1417a0 0x7f97dc141c20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009ae0 tx=0x7f97c8005840 comp rx=0 tx=0).stop 2024-04-25T01:09:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 shutdown_connections 2024-04-25T01:09:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f97c00678c0 0x7f97c0069d80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f97dc0cd3a0 0x7f97dc0cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97dc1417a0 0x7f97dc141c20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 --2- 172.21.15.33:0/2483181581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c8004810 0x7f97dc13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 >> 172.21.15.33:0/2483181581 conn(0x7f97dc0b1810 msgr2=0x7f97dc06caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 shutdown_connections 2024-04-25T01:09:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.720+0000 7f97e3b6b640 1 -- 172.21.15.33:0/2483181581 wait complete. 2024-04-25T01:09:42.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:09:42.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:09:42.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e98fe640 1 Processor -- start 2024-04-25T01:09:42.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e98fe640 1 -- start start 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e98fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92e406eb60 0x7f92e4160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e98fe640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e40cee10 con 0x7f92e4145660 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e98fe640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92e40cf0f0 con 0x7f92e4071170 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e98fe640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f92e4145a40 con 0x7f92e406eb60 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e27fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f92e406eb60 0x7f92e4160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e2ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f92e4145660 0x7f92e415f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39822/0 (socket says 172.21.15.33:39822) 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e2ffd640 1 -- 172.21.15.33:0/289760548 learned_addr learned my addr 172.21.15.33:0/289760548 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e27fc640 1 -- 172.21.15.33:0/289760548 >> v1:172.21.15.138:6789/0 conn(0x7f92e4071170 legacy=0x7f92e40715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e27fc640 1 -- 172.21.15.33:0/289760548 >> v1:172.21.15.33:6789/0 conn(0x7f92e4145660 legacy=0x7f92e415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.868+0000 7f92e27fc640 1 -- 172.21.15.33:0/289760548 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e40cf3d0 con 0x7f92e406eb60 2024-04-25T01:09:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e27fc640 1 --2- 172.21.15.33:0/289760548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e406eb60 0x7f92e4160070 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097c0 tx=0x7f92d801cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f166222b0e454e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e1ffb640 1 -- 172.21.15.33:0/289760548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80b0070 con 0x7f92e406eb60 2024-04-25T01:09:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e1ffb640 1 -- 172.21.15.33:0/289760548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d80ab600 con 0x7f92e406eb60 2024-04-25T01:09:42.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e1ffb640 1 -- 172.21.15.33:0/289760548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80ab9d0 con 0x7f92e406eb60 2024-04-25T01:09:42.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 -- 172.21.15.33:0/289760548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e406eb60 msgr2=0x7f92e4160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 --2- 172.21.15.33:0/289760548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e406eb60 0x7f92e4160070 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097c0 tx=0x7f92d801cec0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 -- 172.21.15.33:0/289760548 shutdown_connections 2024-04-25T01:09:42.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 --2- 172.21.15.33:0/289760548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e406eb60 0x7f92e4160070 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 -- 172.21.15.33:0/289760548 >> 172.21.15.33:0/289760548 conn(0x7f92e40b1810 msgr2=0x7f92e40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:42.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 -- 172.21.15.33:0/289760548 shutdown_connections 2024-04-25T01:09:42.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 -- 172.21.15.33:0/289760548 wait complete. 2024-04-25T01:09:42.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 Processor -- start 2024-04-25T01:09:42.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.872+0000 7f92e98fe640 1 -- start start 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e406eb60 0x7f92e41585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e4071170 0x7f92e4137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e4145660 0x7f92e4133470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4169e30 con 0x7f92e4071170 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92e4169cb0 con 0x7f92e4145660 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f92e4169fb0 con 0x7f92e406eb60 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e4071170 0x7f92e4137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e4071170 0x7f92e4137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53770/0 (socket says 172.21.15.33:53770) 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 -- 172.21.15.33:0/3168277276 learned_addr learned my addr 172.21.15.33:0/3168277276 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:42.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e2ffd640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e406eb60 0x7f92e41585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 -- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e406eb60 msgr2=0x7f92e41585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e406eb60 0x7f92e41585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e37fe640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e4145660 0x7f92e4133470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 -- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e4145660 msgr2=0x7f92e4133470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e4145660 0x7f92e4133470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 -- 172.21.15.33:0/3168277276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92e4138340 con 0x7f92e4071170 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e27fc640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e4071170 0x7f92e4137960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f92d8019070 tx=0x7f92d8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92cbfff640 1 -- 172.21.15.33:0/3168277276 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80b8030 con 0x7f92e4071170 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92cbfff640 1 -- 172.21.15.33:0/3168277276 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92d80b4410 con 0x7f92e4071170 2024-04-25T01:09:42.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92cbfff640 1 -- 172.21.15.33:0/3168277276 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92d80b0040 con 0x7f92e4071170 2024-04-25T01:09:42.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92e4138620 con 0x7f92e4071170 2024-04-25T01:09:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92e4138be0 con 0x7f92e4071170 2024-04-25T01:09:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92e4069040 con 0x7f92e4071170 2024-04-25T01:09:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92cbfff640 1 -- 172.21.15.33:0/3168277276 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f92d80bb070 con 0x7f92e4071170 2024-04-25T01:09:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92cbfff640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f92d00678c0 0x7f92d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92cbfff640 1 -- 172.21.15.33:0/3168277276 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f92d811eec0 con 0x7f92e4071170 2024-04-25T01:09:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.876+0000 7f92e2ffd640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f92d00678c0 0x7f92d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:42.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.880+0000 7f92e2ffd640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f92d00678c0 0x7f92d0069d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92e4139680 tx=0x7f92d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:42.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:42.884+0000 7f92cbfff640 1 -- 172.21.15.33:0/3168277276 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f92d80c3020 con 0x7f92e4071170 2024-04-25T01:09:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92e4072ac0 con 0x7f92e4071170 2024-04-25T01:09:43.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92cbfff640 1 -- 172.21.15.33:0/3168277276 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f92d8016050 con 0x7f92e4071170 2024-04-25T01:09:43.036 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:09:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f92d00678c0 msgr2=0x7f92d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f92d00678c0 0x7f92d0069d80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f92e4139680 tx=0x7f92d4009040 comp rx=0 tx=0).stop 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e4071170 msgr2=0x7f92e4137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e4071170 0x7f92e4137960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f92d8019070 tx=0x7f92d8019200 comp rx=0 tx=0).stop 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 shutdown_connections 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f92d00678c0 0x7f92d0069d80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92e4145660 0x7f92e4133470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f92e4071170 0x7f92e4137960 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 --2- 172.21.15.33:0/3168277276 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f92e406eb60 0x7f92e41585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 >> 172.21.15.33:0/3168277276 conn(0x7f92e40b1810 msgr2=0x7f92e40ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 shutdown_connections 2024-04-25T01:09:43.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.044+0000 7f92e98fe640 1 -- 172.21.15.33:0/3168277276 wait complete. 2024-04-25T01:09:43.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T01:09:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.180+0000 7f11f0f57640 1 Processor -- start 2024-04-25T01:09:43.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.180+0000 7f11f0f57640 1 -- start start 2024-04-25T01:09:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11f0f57640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec1563d0 0x7f11ec15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11f0f57640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f11ec0b5680 0x7f11ec0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11f0f57640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11ec06f020 con 0x7f11ec0b4c50 2024-04-25T01:09:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11f0f57640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f11ec0b6030 con 0x7f11ec0b5680 2024-04-25T01:09:43.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11f0f57640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11ec06f2a0 con 0x7f11ec1563d0 2024-04-25T01:09:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ea59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec1563d0 0x7f11ec15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11e9d9b640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f11ec0b4c50 0x7f11ec15cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:59336/0 (socket says 172.21.15.33:59336) 2024-04-25T01:09:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ead9d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f11ec0b5680 0x7f11ec0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11e9d9b640 1 -- 172.21.15.33:0/3765346906 learned_addr learned my addr 172.21.15.33:0/3765346906 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11e959a640 1 -- 172.21.15.33:0/3765346906 <== mon.1 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182527900 0 0) 0x7f11e0001120 con 0x7f11ec0b4c50 2024-04-25T01:09:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11e959a640 1 -- 172.21.15.33:0/3765346906 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11d4003530 con 0x7f11ec0b4c50 2024-04-25T01:09:43.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ead9d640 1 -- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec1563d0 msgr2=0x7f11ec15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ead9d640 1 --2- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec1563d0 0x7f11ec15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ead9d640 1 -- 172.21.15.33:0/3765346906 >> v1:172.21.15.138:6789/0 conn(0x7f11ec0b4c50 legacy=0x7f11ec15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ead9d640 1 -- 172.21.15.33:0/3765346906 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec06f580 con 0x7f11ec0b5680 2024-04-25T01:09:43.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ea59c640 1 --2- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec1563d0 0x7f11ec15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:09:43.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11ead9d640 1 --2- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec0b5680 0x7f11ec0b5af0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009d90 tx=0x7f11e401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e463639ee0c1df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11e959a640 1 -- 172.21.15.33:0/3765346906 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e40b2070 con 0x7f11ec0b5680 2024-04-25T01:09:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11e959a640 1 -- 172.21.15.33:0/3765346906 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11e40a57a0 con 0x7f11ec0b5680 2024-04-25T01:09:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.184+0000 7f11e959a640 1 -- 172.21.15.33:0/3765346906 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e40a5b50 con 0x7f11ec0b5680 2024-04-25T01:09:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec0b5680 msgr2=0x7f11ec0b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 --2- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec0b5680 0x7f11ec0b5af0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f11e4009d90 tx=0x7f11e401c8e0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- 172.21.15.33:0/3765346906 shutdown_connections 2024-04-25T01:09:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 --2- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec0b5680 0x7f11ec0b5af0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 --2- 172.21.15.33:0/3765346906 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec1563d0 0x7f11ec15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- 172.21.15.33:0/3765346906 >> 172.21.15.33:0/3765346906 conn(0x7f11ec0734c0 msgr2=0x7f11ec09a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- 172.21.15.33:0/3765346906 shutdown_connections 2024-04-25T01:09:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- 172.21.15.33:0/3765346906 wait complete. 2024-04-25T01:09:43.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 Processor -- start 2024-04-25T01:09:43.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- start start 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11ec0b4c50 0x7f11ec14e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec0b5680 0x7f11ec14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec1563d0 0x7f11ec14ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec070ee0 con 0x7f11ec0b4c50 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec070d60 con 0x7f11ec1563d0 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11ec071060 con 0x7f11ec0b5680 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11ec0b4c50 0x7f11ec14e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11ec0b4c50 0x7f11ec14e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53774/0 (socket says 172.21.15.33:53774) 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 -- 172.21.15.33:0/2232006283 learned_addr learned my addr 172.21.15.33:0/2232006283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ead9d640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec1563d0 0x7f11ec14ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11e9d9b640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec0b5680 0x7f11ec14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 -- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec0b5680 msgr2=0x7f11ec14e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec0b5680 0x7f11ec14e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 -- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec1563d0 msgr2=0x7f11ec14ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec1563d0 0x7f11ec14ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 -- 172.21.15.33:0/2232006283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec14f210 con 0x7f11ec0b4c50 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ead9d640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec1563d0 0x7f11ec14ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11ea59c640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11ec0b4c50 0x7f11ec14e250 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00ebe0 tx=0x7f11dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11daffd640 1 -- 172.21.15.33:0/2232006283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc017070 con 0x7f11ec0b4c50 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11daffd640 1 -- 172.21.15.33:0/2232006283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11dc00cc30 con 0x7f11ec0b4c50 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec152530 con 0x7f11ec0b4c50 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11daffd640 1 -- 172.21.15.33:0/2232006283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11dc013470 con 0x7f11ec0b4c50 2024-04-25T01:09:43.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.188+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec152af0 con 0x7f11ec0b4c50 2024-04-25T01:09:43.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.192+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec069040 con 0x7f11ec0b4c50 2024-04-25T01:09:43.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.192+0000 7f11daffd640 1 -- 172.21.15.33:0/2232006283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f11dc01a070 con 0x7f11ec0b4c50 2024-04-25T01:09:43.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.192+0000 7f11daffd640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11cc0678c0 0x7f11cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.192+0000 7f11daffd640 1 -- 172.21.15.33:0/2232006283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f11dc085e00 con 0x7f11ec0b4c50 2024-04-25T01:09:43.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.192+0000 7f11e9d9b640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11cc0678c0 0x7f11cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.192+0000 7f11e9d9b640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11cc0678c0 0x7f11cc069d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11ec1f8120 tx=0x7f11e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:43.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.196+0000 7f11daffd640 1 -- 172.21.15.33:0/2232006283 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f11ec14f210 con 0x7f11ec0b4c50 2024-04-25T01:09:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.356+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f11ec0d2a90 con 0x7f11ec0b4c50 2024-04-25T01:09:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.356+0000 7f11daffd640 1 -- 172.21.15.33:0/2232006283 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f11dc056b00 con 0x7f11ec0b4c50 2024-04-25T01:09:43.350 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:43.350 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-04-25T01:09:43.350 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-04-25T01:09:43.350 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-04-25T01:09:43.351 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-04-25T01:09:43.351 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:43.351 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-04-25T01:09:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11cc0678c0 msgr2=0x7f11cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11cc0678c0 0x7f11cc069d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f11ec1f8120 tx=0x7f11e0009040 comp rx=0 tx=0).stop 2024-04-25T01:09:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11ec0b4c50 msgr2=0x7f11ec14e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11ec0b4c50 0x7f11ec14e250 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00ebe0 tx=0x7f11dc00c520 comp rx=0 tx=0).stop 2024-04-25T01:09:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 shutdown_connections 2024-04-25T01:09:43.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11cc0678c0 0x7f11cc069d80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11ec1563d0 0x7f11ec14ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11ec0b5680 0x7f11ec14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 --2- 172.21.15.33:0/2232006283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11ec0b4c50 0x7f11ec14e250 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 >> 172.21.15.33:0/2232006283 conn(0x7f11ec0734c0 msgr2=0x7f11ec0b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 shutdown_connections 2024-04-25T01:09:43.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.360+0000 7f11f0f57640 1 -- 172.21.15.33:0/2232006283 wait complete. 2024-04-25T01:09:43.368 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T01:09:43.368 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T01:09:43.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9332618640 1 Processor -- start 2024-04-25T01:09:43.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9332618640 1 -- start start 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9332618640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f932c0cf470 0x7f932c0d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9332618640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c15e630 con 0x7f932c0d2970 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9332618640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c0cf970 con 0x7f932c15e250 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9332618640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f932c0d4d90 con 0x7f932c0cf470 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f932b7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f932c0cf470 0x7f932c0d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9330bb5640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f932c0d2970 0x7f932c0b3d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39836/0 (socket says 172.21.15.33:39836) 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.504+0000 7f9330bb5640 1 -- 172.21.15.33:0/4156088677 learned_addr learned my addr 172.21.15.33:0/4156088677 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f932b7fe640 1 -- 172.21.15.33:0/4156088677 >> v1:172.21.15.138:6789/0 conn(0x7f932c15e250 legacy=0x7f932c0d4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f932b7fe640 1 -- 172.21.15.33:0/4156088677 >> v1:172.21.15.33:6789/0 conn(0x7f932c0d2970 legacy=0x7f932c0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f932b7fe640 1 -- 172.21.15.33:0/4156088677 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c0cfc50 con 0x7f932c0cf470 2024-04-25T01:09:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f932b7fe640 1 --2- 172.21.15.33:0/4156088677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0cf470 0x7f932c0d2430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097c0 tx=0x7f931c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8566a556171fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f932affd640 1 -- 172.21.15.33:0/4156088677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0b5070 con 0x7f932c0cf470 2024-04-25T01:09:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f932affd640 1 -- 172.21.15.33:0/4156088677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c0a8950 con 0x7f932c0cf470 2024-04-25T01:09:43.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f932affd640 1 -- 172.21.15.33:0/4156088677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0a8d20 con 0x7f932c0cf470 2024-04-25T01:09:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f9332618640 1 -- 172.21.15.33:0/4156088677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0cf470 msgr2=0x7f932c0d2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f9332618640 1 --2- 172.21.15.33:0/4156088677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0cf470 0x7f932c0d2430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f931c0097c0 tx=0x7f931c01ec50 comp rx=0 tx=0).stop 2024-04-25T01:09:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f9332618640 1 -- 172.21.15.33:0/4156088677 shutdown_connections 2024-04-25T01:09:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f9332618640 1 --2- 172.21.15.33:0/4156088677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0cf470 0x7f932c0d2430 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f9332618640 1 -- 172.21.15.33:0/4156088677 >> 172.21.15.33:0/4156088677 conn(0x7f932c09d120 msgr2=0x7f932c0d3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f9332618640 1 -- 172.21.15.33:0/4156088677 shutdown_connections 2024-04-25T01:09:43.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.508+0000 7f9332618640 1 -- 172.21.15.33:0/4156088677 wait complete. 2024-04-25T01:09:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 Processor -- start 2024-04-25T01:09:43.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 -- start start 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c0cf470 0x7f932c14b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0d2970 0x7f932c160a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932c15e250 0x7f932c160f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0b37f0 con 0x7f932c0d2970 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f932c0b3510 con 0x7f932c0cf470 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f932c0b4420 con 0x7f932c15e250 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0d2970 0x7f932c160a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0d2970 0x7f932c160a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53792/0 (socket says 172.21.15.33:53792) 2024-04-25T01:09:43.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 -- 172.21.15.33:0/53359962 learned_addr learned my addr 172.21.15.33:0/53359962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932bfff640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c0cf470 0x7f932c14b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 -- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932c15e250 msgr2=0x7f932c160f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932c15e250 0x7f932c160f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 -- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c0cf470 msgr2=0x7f932c14b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c0cf470 0x7f932c14b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 -- 172.21.15.33:0/53359962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c14b790 con 0x7f932c0d2970 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932bfff640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c0cf470 0x7f932c14b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932b7fe640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0d2970 0x7f932c160a00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f931c01b860 tx=0x7f931c01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9328ff9640 1 -- 172.21.15.33:0/53359962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0bf070 con 0x7f932c0d2970 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9328ff9640 1 -- 172.21.15.33:0/53359962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c0b0900 con 0x7f932c0d2970 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c1f4750 con 0x7f932c0d2970 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9328ff9640 1 -- 172.21.15.33:0/53359962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0b5040 con 0x7f932c0d2970 2024-04-25T01:09:43.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c14dfb0 con 0x7f932c0d2970 2024-04-25T01:09:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f932c069040 con 0x7f932c0d2970 2024-04-25T01:09:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9328ff9640 1 -- 172.21.15.33:0/53359962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f931c0c2070 con 0x7f932c0d2970 2024-04-25T01:09:43.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9328ff9640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93180678c0 0x7f9318069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f9328ff9640 1 -- 172.21.15.33:0/53359962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f931c122330 con 0x7f932c0d2970 2024-04-25T01:09:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.512+0000 7f932bfff640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93180678c0 0x7f9318069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:43.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.516+0000 7f932bfff640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93180678c0 0x7f9318069d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f932000a850 tx=0x7f9320008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:43.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.520+0000 7f9328ff9640 1 -- 172.21.15.33:0/53359962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f931c0187b0 con 0x7f932c0d2970 2024-04-25T01:09:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.680+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f932c0673b0 con 0x7f932c0d2970 2024-04-25T01:09:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9328ff9640 1 -- 172.21.15.33:0/53359962 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f931c0f3030 con 0x7f932c0d2970 2024-04-25T01:09:43.888 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93180678c0 msgr2=0x7f9318069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93180678c0 0x7f9318069d80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f932000a850 tx=0x7f9320008040 comp rx=0 tx=0).stop 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0d2970 msgr2=0x7f932c160a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0d2970 0x7f932c160a00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f931c01b860 tx=0x7f931c01b9f0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 shutdown_connections 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f93180678c0 0x7f9318069d80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f932c15e250 0x7f932c160f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f932c0d2970 0x7f932c160a00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 --2- 172.21.15.33:0/53359962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f932c0cf470 0x7f932c14b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 >> 172.21.15.33:0/53359962 conn(0x7f932c09d120 msgr2=0x7f932c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:43.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 shutdown_connections 2024-04-25T01:09:43.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:43.896+0000 7f9332618640 1 -- 172.21.15.33:0/53359962 wait complete. 2024-04-25T01:09:43.914 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T01:09:44.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f856844a640 1 Processor -- start 2024-04-25T01:09:44.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f856844a640 1 -- start start 2024-04-25T01:09:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f856844a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 0x7f856015ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f856844a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f856015d5d0 0x7f856015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f856844a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85600b05a0 con 0x7f85600b5580 2024-04-25T01:09:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f856844a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f856015d380 con 0x7f856015d5d0 2024-04-25T01:09:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f856844a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85600af500 con 0x7f85600b50d0 2024-04-25T01:09:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85661e6640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f85600b5580 0x7f85600b5960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39844/0 (socket says 172.21.15.33:39844) 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85661e6640 1 -- 172.21.15.33:0/1202865356 learned_addr learned my addr 172.21.15.33:0/1202865356 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85669e7640 1 --2- 172.21.15.33:0/1202865356 >> v2:172.21.15.138:3300/0 conn(0x7f856015d5d0 0x7f856015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85659e5640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 0x7f856015ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2632251146 0 0) 0x7f855c001120 con 0x7f85600b5580 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f854c003530 con 0x7f85600b5580 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3907656182 0 0) 0x7f854c003530 con 0x7f85600b5580 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 msgr2=0x7f856015ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 0x7f856015ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 >> v2:172.21.15.138:3300/0 conn(0x7f856015d5d0 msgr2=0x7f856015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 --2- 172.21.15.33:0/1202865356 >> v2:172.21.15.138:3300/0 conn(0x7f856015d5d0 0x7f856015da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85600af7e0 con 0x7f85600b5580 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f855c001850 con 0x7f85600b5580 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 >> v1:172.21.15.33:6789/0 conn(0x7f85600b5580 legacy=0x7f85600b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f854c0039c0 0x7f854c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 0x7f854c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c004d50 0x7f854c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f856015d380 con 0x7f854c004d50 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85600af500 con 0x7f854c0039c0 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f854c00b9d0 con 0x7f85600b50d0 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85669e7640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c004d50 0x7f854c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85659e5640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 0x7f854c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85669e7640 1 -- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 msgr2=0x7f854c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85661e6640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f854c0039c0 0x7f854c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85669e7640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f85600b50d0 0x7f854c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85669e7640 1 -- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f854c0039c0 msgr2=0x7f854c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85669e7640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f854c0039c0 0x7f854c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85669e7640 1 -- 172.21.15.33:0/1202865356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c003530 con 0x7f854c004d50 2024-04-25T01:09:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.056+0000 7f85661e6640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f854c0039c0 0x7f854c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f85669e7640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c004d50 0x7f854c0051d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8558009fc0 tx=0x7f855801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dbb3646c4eabe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85580b2070 con 0x7f854c004d50 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85580a5cd0 con 0x7f854c004d50 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f85651e4640 1 -- 172.21.15.33:0/1202865356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558009c10 con 0x7f854c004d50 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 -- 172.21.15.33:0/1202865356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c004d50 msgr2=0x7f854c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 --2- 172.21.15.33:0/1202865356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f854c004d50 0x7f854c0051d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8558009fc0 tx=0x7f855801ccc0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f85669e7640 1 -- 172.21.15.33:0/1202865356 reap_dead start 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 -- 172.21.15.33:0/1202865356 shutdown_connections 2024-04-25T01:09:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 -- 172.21.15.33:0/1202865356 >> 172.21.15.33:0/1202865356 conn(0x7f856009a9d0 msgr2=0x7f856009ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 -- 172.21.15.33:0/1202865356 shutdown_connections 2024-04-25T01:09:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 -- 172.21.15.33:0/1202865356 wait complete. 2024-04-25T01:09:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 Processor -- start 2024-04-25T01:09:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 -- start start 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.060+0000 7f856844a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85600b3520 0x7f8560157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85600b3900 0x7f8560158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8560147290 0x7f8560147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f85600b0080 con 0x7f85600b3900 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85600afda0 con 0x7f85600b3520 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f85600af1b0 con 0x7f8560147290 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85600b3900 0x7f8560158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85600b3900 0x7f8560158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53802/0 (socket says 172.21.15.33:53802) 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 -- 172.21.15.33:0/2838050233 learned_addr learned my addr 172.21.15.33:0/2838050233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:44.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85661e6640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85600b3520 0x7f8560157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85669e7640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8560147290 0x7f8560147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 -- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8560147290 msgr2=0x7f8560147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8560147290 0x7f8560147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 -- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85600b3520 msgr2=0x7f8560157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85600b3520 0x7f8560157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 -- 172.21.15.33:0/2838050233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8560158780 con 0x7f85600b3900 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85661e6640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85600b3520 0x7f8560157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85659e5640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85600b3900 0x7f8560158150 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8550010850 tx=0x7f8550010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f8556ffd640 1 -- 172.21.15.33:0/2838050233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855000f040 con 0x7f85600b3900 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f8556ffd640 1 -- 172.21.15.33:0/2838050233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f855000eb60 con 0x7f85600b3900 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f8556ffd640 1 -- 172.21.15.33:0/2838050233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f855001c400 con 0x7f85600b3900 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8560158a60 con 0x7f85600b3900 2024-04-25T01:09:44.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85600b1a60 con 0x7f85600b3900 2024-04-25T01:09:44.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8560069040 con 0x7f85600b3900 2024-04-25T01:09:44.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f8556ffd640 1 -- 172.21.15.33:0/2838050233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f855001c5a0 con 0x7f85600b3900 2024-04-25T01:09:44.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f8556ffd640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85440678c0 0x7f8544069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:44.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f8556ffd640 1 -- 172.21.15.33:0/2838050233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f85500861d0 con 0x7f85600b3900 2024-04-25T01:09:44.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.064+0000 7f85661e6640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85440678c0 0x7f8544069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:44.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.068+0000 7f85661e6640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85440678c0 0x7f8544069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f855c0019c0 tx=0x7f855c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:44.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.068+0000 7f8556ffd640 1 -- 172.21.15.33:0/2838050233 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8550018030 con 0x7f85600b3900 2024-04-25T01:09:44.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.228+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f85600d1930 con 0x7f85600b3900 2024-04-25T01:09:44.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.900+0000 7f8556ffd640 1 -- 172.21.15.33:0/2838050233 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f8550056df0 con 0x7f85600b3900 2024-04-25T01:09:44.893 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-04-25T01:09:44.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85440678c0 msgr2=0x7f8544069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85440678c0 0x7f8544069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f855c0019c0 tx=0x7f855c009040 comp rx=0 tx=0).stop 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85600b3900 msgr2=0x7f8560158150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85600b3900 0x7f8560158150 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8550010850 tx=0x7f8550010d10 comp rx=0 tx=0).stop 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 shutdown_connections 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f85440678c0 0x7f8544069d80 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f855c0019c0 tx=0x7f855c009040 comp rx=0 tx=0).stop 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8560147290 0x7f8560147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85600b3900 0x7f8560158150 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 --2- 172.21.15.33:0/2838050233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85600b3520 0x7f8560157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:44.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 >> 172.21.15.33:0/2838050233 conn(0x7f856009a9d0 msgr2=0x7f85601550b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:44.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 shutdown_connections 2024-04-25T01:09:44.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:44.904+0000 7f856844a640 1 -- 172.21.15.33:0/2838050233 wait complete. 2024-04-25T01:09:44.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T01:09:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1baf48640 1 Processor -- start 2024-04-25T01:09:45.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1baf48640 1 -- start start 2024-04-25T01:09:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1baf48640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fa1b406eb60 0x7fa1b4160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1baf48640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 0x7fa1b4071600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1baf48640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1b40cee10 con 0x7fa1b4145660 2024-04-25T01:09:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1baf48640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fa1b4145a40 con 0x7fa1b406eb60 2024-04-25T01:09:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1baf48640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b4070ce0 con 0x7fa1b4071170 2024-04-25T01:09:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b8ce4640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fa1b4145660 0x7fa1b415f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39854/0 (socket says 172.21.15.33:39854) 2024-04-25T01:09:45.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b8ce4640 1 -- 172.21.15.33:0/1360724773 learned_addr learned my addr 172.21.15.33:0/1360724773 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b3fff640 1 --2- 172.21.15.33:0/1360724773 >> v2:172.21.15.138:3300/0 conn(0x7fa1b406eb60 0x7fa1b4160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b94e5640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 0x7fa1b4071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4250917650 0 0) 0x7fa1b40cee10 con 0x7fa1b4145660 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa19c003530 con 0x7fa1b4145660 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 248388375 0 0) 0x7fa19c003530 con 0x7fa1b4145660 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 msgr2=0x7fa1b4071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 0x7fa1b4071600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 >> v2:172.21.15.138:3300/0 conn(0x7fa1b406eb60 msgr2=0x7fa1b4160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 --2- 172.21.15.33:0/1360724773 >> v2:172.21.15.138:3300/0 conn(0x7fa1b406eb60 0x7fa1b4160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b40cf0f0 con 0x7fa1b4145660 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fa1a4001680 con 0x7fa1b4145660 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 >> v1:172.21.15.33:6789/0 conn(0x7fa1b4145660 legacy=0x7fa1b415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 0x7fa19c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa19c003f40 0x7fa19c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa19c004d50 0x7fa19c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4070ce0 con 0x7fa19c004d50 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c00b9d0 con 0x7fa19c003f40 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa19c00bb50 con 0x7fa1b4071170 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b94e5640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa19c004d50 0x7fa19c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b8ce4640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 0x7fa19c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b3fff640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa19c003f40 0x7fa19c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b94e5640 1 -- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 msgr2=0x7fa19c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b94e5640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b4071170 0x7fa19c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b94e5640 1 -- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa19c003f40 msgr2=0x7fa19c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b94e5640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa19c003f40 0x7fa19c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.072+0000 7fa1b94e5640 1 -- 172.21.15.33:0/1360724773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c003530 con 0x7fa19c004d50 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1b94e5640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa19c004d50 0x7fa19c0051d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009fc0 tx=0x7fa1ac01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2baf9fb581085e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac0b2070 con 0x7fa19c004d50 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ac0a5c90 con 0x7fa19c004d50 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1b37fe640 1 -- 172.21.15.33:0/1360724773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ac009c10 con 0x7fa19c004d50 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- 172.21.15.33:0/1360724773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa19c004d50 msgr2=0x7fa19c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 --2- 172.21.15.33:0/1360724773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa19c004d50 0x7fa19c0051d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009fc0 tx=0x7fa1ac01cc20 comp rx=0 tx=0).stop 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1b94e5640 1 -- 172.21.15.33:0/1360724773 reap_dead start 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- 172.21.15.33:0/1360724773 shutdown_connections 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- 172.21.15.33:0/1360724773 >> 172.21.15.33:0/1360724773 conn(0x7fa1b40b1810 msgr2=0x7fa1b40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- 172.21.15.33:0/1360724773 shutdown_connections 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- 172.21.15.33:0/1360724773 wait complete. 2024-04-25T01:09:45.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 Processor -- start 2024-04-25T01:09:45.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- start start 2024-04-25T01:09:45.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b40cc260 0x7fa1b40ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1b40cc640 0x7fa1b40ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1b40a9eb0 0x7fa1b40aa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b406efa0 con 0x7fa1b40cc640 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b40cf6b0 con 0x7fa1b40a9eb0 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.076+0000 7fa1baf48640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b41605b0 con 0x7fa1b40cc260 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1b40cc640 0x7fa1b40ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1b40cc640 0x7fa1b40ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53826/0 (socket says 172.21.15.33:53826) 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 -- 172.21.15.33:0/4268760516 learned_addr learned my addr 172.21.15.33:0/4268760516 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b94e5640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1b40a9eb0 0x7fa1b40aa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b8ce4640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b40cc260 0x7fa1b40ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 -- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b40cc260 msgr2=0x7fa1b40ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b40cc260 0x7fa1b40ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 -- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1b40a9eb0 msgr2=0x7fa1b40aa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1b40a9eb0 0x7fa1b40aa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 -- 172.21.15.33:0/4268760516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b41f1570 con 0x7fa1b40cc640 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b94e5640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1b40a9eb0 0x7fa1b40aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b3fff640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1b40cc640 0x7fa1b40ae780 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8011640 tx=0x7fa1a8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b17fa640 1 -- 172.21.15.33:0/4268760516 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800f920 con 0x7fa1b40cc640 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b17fa640 1 -- 172.21.15.33:0/4268760516 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a800fac0 con 0x7fa1b40cc640 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b40aac00 con 0x7fa1b40cc640 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b17fa640 1 -- 172.21.15.33:0/4268760516 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a800fe70 con 0x7fa1b40cc640 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b40ac150 con 0x7fa1b40cc640 2024-04-25T01:09:45.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b4069040 con 0x7fa1b40cc640 2024-04-25T01:09:45.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b17fa640 1 -- 172.21.15.33:0/4268760516 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa1a8015460 con 0x7fa1b40cc640 2024-04-25T01:09:45.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b17fa640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa1a00678c0 0x7fa1a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:45.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b17fa640 1 -- 172.21.15.33:0/4268760516 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7fa1a8086670 con 0x7fa1b40cc640 2024-04-25T01:09:45.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.080+0000 7fa1b8ce4640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa1a00678c0 0x7fa1a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:45.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.084+0000 7fa1b8ce4640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa1a00678c0 0x7fa1a0069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4001ec0 tx=0x7fa1a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:45.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.084+0000 7fa1b17fa640 1 -- 172.21.15.33:0/4268760516 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fa1a8019030 con 0x7fa1b40cc640 2024-04-25T01:09:45.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.284+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa1b4072ac0 con 0x7fa1b40cc640 2024-04-25T01:09:45.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.936+0000 7fa1b17fa640 1 -- 172.21.15.33:0/4268760516 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa1a80571f0 con 0x7fa1b40cc640 2024-04-25T01:09:45.937 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T01:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.948+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa1a00678c0 msgr2=0x7fa1a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.948+0000 7fa1baf48640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa1a00678c0 0x7fa1a0069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4001ec0 tx=0x7fa1a4009040 comp rx=0 tx=0).stop 2024-04-25T01:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1b40cc640 msgr2=0x7fa1b40ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1b40cc640 0x7fa1b40ae780 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8011640 tx=0x7fa1a8011b00 comp rx=0 tx=0).stop 2024-04-25T01:09:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 shutdown_connections 2024-04-25T01:09:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fa1a00678c0 0x7fa1a0069d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1b40a9eb0 0x7fa1b40aa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1b40cc640 0x7fa1b40ae780 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 --2- 172.21.15.33:0/4268760516 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fa1b40cc260 0x7fa1b40ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 >> 172.21.15.33:0/4268760516 conn(0x7fa1b40b1810 msgr2=0x7fa1b40b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 shutdown_connections 2024-04-25T01:09:45.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:45.952+0000 7fa1baf48640 1 -- 172.21.15.33:0/4268760516 wait complete. 2024-04-25T01:09:45.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:46.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab32359640 1 Processor -- start 2024-04-25T01:09:46.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab32359640 1 -- start start 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab32359640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fab2c15fef0 0x7fab2c1602d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab32359640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab2c0703a0 con 0x7fab2c0707c0 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab32359640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab2c160940 con 0x7fab2c070c20 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab32359640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fab2c0d28f0 con 0x7fab2c15fef0 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2b7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fab2c0707c0 0x7fab2c0d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39864/0 (socket says 172.21.15.33:39864) 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2b7fe640 1 -- 172.21.15.33:0/3516127759 learned_addr learned my addr 172.21.15.33:0/3516127759 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228676295 0 0) 0x7fab2c0703a0 con 0x7fab2c0707c0 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab18003530 con 0x7fab2c0707c0 2024-04-25T01:09:46.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2bfff640 1 --2- 172.21.15.33:0/3516127759 >> v2:172.21.15.138:3300/0 conn(0x7fab2c15fef0 0x7fab2c1602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4036399789 0 0) 0x7fab18003530 con 0x7fab2c0707c0 2024-04-25T01:09:46.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 >> v2:172.21.15.138:3300/0 conn(0x7fab2c15fef0 msgr2=0x7fab2c1602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 --2- 172.21.15.33:0/3516127759 >> v2:172.21.15.138:3300/0 conn(0x7fab2c15fef0 0x7fab2c1602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 >> v1:172.21.15.138:6789/0 conn(0x7fab2c070c20 legacy=0x7fab2c0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c0d0990 con 0x7fab2c0707c0 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fab14001680 con 0x7fab2c0707c0 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 >> v1:172.21.15.33:6789/0 conn(0x7fab2c0707c0 legacy=0x7fab2c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18003a10 0x7fab18003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab18004810 0x7fab18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab180052f0 0x7fab1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c0d28f0 con 0x7fab18003a10 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab1800c1f0 con 0x7fab18004810 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fab1800c3f0 con 0x7fab180052f0 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2bfff640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18003a10 0x7fab18003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2b7fe640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab18004810 0x7fab18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2bfff640 1 -- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab180052f0 msgr2=0x7fab1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2bfff640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab180052f0 0x7fab1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2bfff640 1 -- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab18004810 msgr2=0x7fab18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2bfff640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab18004810 0x7fab18004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2bfff640 1 -- 172.21.15.33:0/3516127759 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab18003530 con 0x7fab18003a10 2024-04-25T01:09:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab308f6640 1 -- 172.21.15.33:0/3516127759 reap_dead start 2024-04-25T01:09:46.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.108+0000 7fab2b7fe640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab18004810 0x7fab18004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab2bfff640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18003a10 0x7fab18003e30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009fc0 tx=0x7fab1c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6599723d63d1ed17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0b2070 con 0x7fab18003a10 2024-04-25T01:09:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1c0a5cd0 con 0x7fab18003a10 2024-04-25T01:09:46.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab2affd640 1 -- 172.21.15.33:0/3516127759 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c009c10 con 0x7fab18003a10 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 -- 172.21.15.33:0/3516127759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18003a10 msgr2=0x7fab18003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18003a10 0x7fab18003e30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009fc0 tx=0x7fab1c01ccc0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 -- 172.21.15.33:0/3516127759 shutdown_connections 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 --2- 172.21.15.33:0/3516127759 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab18003a10 0x7fab18003e30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 -- 172.21.15.33:0/3516127759 >> 172.21.15.33:0/3516127759 conn(0x7fab2c0d1ba0 msgr2=0x7fab2c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 -- 172.21.15.33:0/3516127759 shutdown_connections 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 -- 172.21.15.33:0/3516127759 wait complete. 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 Processor -- start 2024-04-25T01:09:46.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.112+0000 7fab32359640 1 -- start start 2024-04-25T01:09:46.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab18003a10 0x7fab2c1f8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c146bf0 0x7fab2c0c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab2c0c7dd0 0x7fab2c156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c15efc0 con 0x7fab2c146bf0 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fab2c0afda0 con 0x7fab2c0c7dd0 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fab2c15f140 con 0x7fab18003a10 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c146bf0 0x7fab2c0c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c146bf0 0x7fab2c0c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53848/0 (socket says 172.21.15.33:53848) 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 -- 172.21.15.33:0/4045892071 learned_addr learned my addr 172.21.15.33:0/4045892071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2bfff640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab18003a10 0x7fab2c1f8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab308f6640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab2c0c7dd0 0x7fab2c156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 -- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab18003a10 msgr2=0x7fab2c1f8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab18003a10 0x7fab2c1f8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 -- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab2c0c7dd0 msgr2=0x7fab2c156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab2c0c7dd0 0x7fab2c156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 -- 172.21.15.33:0/4045892071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2c0c8fa0 con 0x7fab2c146bf0 2024-04-25T01:09:46.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab2b7fe640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c146bf0 0x7fab2c0c7890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fab14007fd0 tx=0x7fab1400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab28ff9640 1 -- 172.21.15.33:0/4045892071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab14019070 con 0x7fab2c146bf0 2024-04-25T01:09:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab28ff9640 1 -- 172.21.15.33:0/4045892071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab140072d0 con 0x7fab2c146bf0 2024-04-25T01:09:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab28ff9640 1 -- 172.21.15.33:0/4045892071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab140047f0 con 0x7fab2c146bf0 2024-04-25T01:09:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2c0c9280 con 0x7fab2c146bf0 2024-04-25T01:09:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab2c0c9840 con 0x7fab2c146bf0 2024-04-25T01:09:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab2c0c81e0 con 0x7fab2c146bf0 2024-04-25T01:09:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab28ff9640 1 -- 172.21.15.33:0/4045892071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab14004990 con 0x7fab2c146bf0 2024-04-25T01:09:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.116+0000 7fab28ff9640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab0c0678c0 0x7fab0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.120+0000 7fab28ff9640 1 -- 172.21.15.33:0/4045892071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fab140856c0 con 0x7fab2c146bf0 2024-04-25T01:09:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.120+0000 7fab2bfff640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab0c0678c0 0x7fab0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.120+0000 7fab2bfff640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab0c0678c0 0x7fab0c069d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0af000 tx=0x7fab1c0a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.124+0000 7fab28ff9640 1 -- 172.21.15.33:0/4045892071 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fab2c0c81e0 con 0x7fab2c146bf0 2024-04-25T01:09:46.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.284+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fab2c069040 con 0x7fab2c146bf0 2024-04-25T01:09:46.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab28ff9640 1 -- 172.21.15.33:0/4045892071 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16943 (secure 0 0 0) 0x7fab14056100 con 0x7fab2c146bf0 2024-04-25T01:09:46.281 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:46.281 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":38,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:45.287963+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:09:43.725950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T01:09:44.271067+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab0c0678c0 msgr2=0x7fab0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab0c0678c0 0x7fab0c069d80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0af000 tx=0x7fab1c0a7040 comp rx=0 tx=0).stop 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c146bf0 msgr2=0x7fab2c0c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c146bf0 0x7fab2c0c7890 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fab14007fd0 tx=0x7fab1400dc00 comp rx=0 tx=0).stop 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 shutdown_connections 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fab0c0678c0 0x7fab0c069d80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab2c0c7dd0 0x7fab2c156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab2c146bf0 0x7fab2c0c7890 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 --2- 172.21.15.33:0/4045892071 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fab18003a10 0x7fab2c1f8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.288+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 >> 172.21.15.33:0/4045892071 conn(0x7fab2c0d1ba0 msgr2=0x7fab2c0aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.292+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 shutdown_connections 2024-04-25T01:09:46.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.292+0000 7fab32359640 1 -- 172.21.15.33:0/4045892071 wait complete. 2024-04-25T01:09:46.301 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:46.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.432+0000 7fef11c1b640 1 Processor -- start 2024-04-25T01:09:46.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.432+0000 7fef11c1b640 1 -- start start 2024-04-25T01:09:46.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef11c1b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fef0c15f220 0x7fef0c06f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef11c1b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fef0c06e050 0x7fef0c06fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef11c1b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0c0b6060 con 0x7fef0c06dc70 2024-04-25T01:09:46.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef11c1b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c0b6340 con 0x7fef0c15f220 2024-04-25T01:09:46.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef11c1b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fef0c0b6620 con 0x7fef0c06e050 2024-04-25T01:09:46.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fef0c15f220 0x7fef0c06f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fef0c15f220 0x7fef0c06f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53850/0 (socket says 172.21.15.33:53850) 2024-04-25T01:09:46.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0affd640 1 -- 172.21.15.33:0/4264106304 learned_addr learned my addr 172.21.15.33:0/4264106304 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:46.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0bfff640 1 --2- 172.21.15.33:0/4264106304 >> v2:172.21.15.138:3300/0 conn(0x7fef0c06e050 0x7fef0c06fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0affd640 1 -- 172.21.15.33:0/4264106304 >> v2:172.21.15.138:3300/0 conn(0x7fef0c06e050 msgr2=0x7fef0c06fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:46.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0affd640 1 --2- 172.21.15.33:0/4264106304 >> v2:172.21.15.138:3300/0 conn(0x7fef0c06e050 0x7fef0c06fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0affd640 1 -- 172.21.15.33:0/4264106304 >> v1:172.21.15.138:6789/0 conn(0x7fef0c06dc70 legacy=0x7fef0c0b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:46.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.436+0000 7fef0affd640 1 -- 172.21.15.33:0/4264106304 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c0b6900 con 0x7fef0c15f220 2024-04-25T01:09:46.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef0affd640 1 --2- 172.21.15.33:0/4264106304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c15f220 0x7fef0c06f8a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609dfd37c64043e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef0a7fc640 1 -- 172.21.15.33:0/4264106304 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0ab5f0 con 0x7fef0c15f220 2024-04-25T01:09:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef0a7fc640 1 -- 172.21.15.33:0/4264106304 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc0b0070 con 0x7fef0c15f220 2024-04-25T01:09:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef0a7fc640 1 -- 172.21.15.33:0/4264106304 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0ab9b0 con 0x7fef0c15f220 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 -- 172.21.15.33:0/4264106304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c15f220 msgr2=0x7fef0c06f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 --2- 172.21.15.33:0/4264106304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c15f220 0x7fef0c06f8a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feefc009870 tx=0x7feefc01c4e0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 -- 172.21.15.33:0/4264106304 shutdown_connections 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 --2- 172.21.15.33:0/4264106304 >> v2:172.21.15.138:3300/0 conn(0x7fef0c06e050 0x7fef0c06fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 --2- 172.21.15.33:0/4264106304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c15f220 0x7fef0c06f8a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 -- 172.21.15.33:0/4264106304 >> 172.21.15.33:0/4264106304 conn(0x7fef0c0734c0 msgr2=0x7fef0c09a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 -- 172.21.15.33:0/4264106304 shutdown_connections 2024-04-25T01:09:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.440+0000 7fef11c1b640 1 -- 172.21.15.33:0/4264106304 wait complete. 2024-04-25T01:09:46.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 Processor -- start 2024-04-25T01:09:46.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 -- start start 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c06dc70 0x7fef0c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef0c06e050 0x7fef0c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c15f220 0x7fef0c1599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c06cec0 con 0x7fef0c06dc70 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef0c06cd40 con 0x7fef0c15f220 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fef0c06d040 con 0x7fef0c06e050 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c06dc70 0x7fef0c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c06dc70 0x7fef0c153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53856/0 (socket says 172.21.15.33:53856) 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 -- 172.21.15.33:0/1375524342 learned_addr learned my addr 172.21.15.33:0/1375524342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:46.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0bfff640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c15f220 0x7fef0c1599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0affd640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef0c06e050 0x7fef0c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 -- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef0c06e050 msgr2=0x7fef0c154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef0c06e050 0x7fef0c154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 -- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c15f220 msgr2=0x7fef0c1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c15f220 0x7fef0c1599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 -- 172.21.15.33:0/1375524342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0c14a9b0 con 0x7fef0c06dc70 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef0b7fe640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c06dc70 0x7fef0c153f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feef400c450 tx=0x7feef400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef03fff640 1 -- 172.21.15.33:0/1375524342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4012830 con 0x7fef0c06dc70 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef03fff640 1 -- 172.21.15.33:0/1375524342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef4017070 con 0x7fef0c06dc70 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0c14ac90 con 0x7fef0c06dc70 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef03fff640 1 -- 172.21.15.33:0/1375524342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4012c30 con 0x7fef0c06dc70 2024-04-25T01:09:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0c14b250 con 0x7fef0c06dc70 2024-04-25T01:09:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.444+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef0c069040 con 0x7fef0c06dc70 2024-04-25T01:09:46.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.448+0000 7fef03fff640 1 -- 172.21.15.33:0/1375524342 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feef401c460 con 0x7fef0c06dc70 2024-04-25T01:09:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.448+0000 7fef03fff640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7feeec0678f0 0x7feeec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.448+0000 7fef03fff640 1 -- 172.21.15.33:0/1375524342 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7feef4085c20 con 0x7fef0c06dc70 2024-04-25T01:09:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.448+0000 7fef0affd640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7feeec0678f0 0x7feeec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.448+0000 7fef0affd640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7feeec0678f0 0x7feeec069db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feefc0af000 tx=0x7feefc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.452+0000 7fef03fff640 1 -- 172.21.15.33:0/1375524342 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feef40565d0 con 0x7fef0c06dc70 2024-04-25T01:09:46.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.616+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef0c06bfe0 con 0x7fef0c06dc70 2024-04-25T01:09:46.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.620+0000 7fef03fff640 1 -- 172.21.15.33:0/1375524342 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16943 (secure 0 0 0) 0x7feef4055ed0 con 0x7fef0c06dc70 2024-04-25T01:09:46.613 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:46.613 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":38,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:45.287963+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:09:43.725950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T01:09:44.271067+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7feeec0678f0 msgr2=0x7feeec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7feeec0678f0 0x7feeec069db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feefc0af000 tx=0x7feefc008040 comp rx=0 tx=0).stop 2024-04-25T01:09:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c06dc70 msgr2=0x7fef0c153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c06dc70 0x7fef0c153f40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feef400c450 tx=0x7feef400c910 comp rx=0 tx=0).stop 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 shutdown_connections 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7feeec0678f0 0x7feeec069db0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef0c15f220 0x7fef0c1599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fef0c06e050 0x7fef0c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 --2- 172.21.15.33:0/1375524342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef0c06dc70 0x7fef0c153f40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 >> 172.21.15.33:0/1375524342 conn(0x7fef0c0734c0 msgr2=0x7fef0c06f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 shutdown_connections 2024-04-25T01:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.624+0000 7fef11c1b640 1 -- 172.21.15.33:0/1375524342 wait complete. 2024-04-25T01:09:46.634 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T01:09:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe3deb6640 1 Processor -- start 2024-04-25T01:09:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe3deb6640 1 -- start start 2024-04-25T01:09:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe3deb6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe38145660 0x7efe3806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe3deb6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7efe381604b0 0x7efe38071170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe3deb6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe380cee10 con 0x7efe3815f960 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe3deb6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7efe3806f0a0 con 0x7efe38145660 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe3deb6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7efe38145a40 con 0x7efe381604b0 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe377fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7efe38145660 0x7efe3806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe36ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7efe3815f960 0x7efe3815fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39872/0 (socket says 172.21.15.33:39872) 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe36ffd640 1 -- 172.21.15.33:0/539630619 learned_addr learned my addr 172.21.15.33:0/539630619 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe37fff640 1 --2- 172.21.15.33:0/539630619 >> v2:172.21.15.138:3300/0 conn(0x7efe381604b0 0x7efe38071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe377fe640 1 -- 172.21.15.33:0/539630619 >> v2:172.21.15.138:3300/0 conn(0x7efe381604b0 msgr2=0x7efe38071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe377fe640 1 --2- 172.21.15.33:0/539630619 >> v2:172.21.15.138:3300/0 conn(0x7efe381604b0 0x7efe38071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe377fe640 1 -- 172.21.15.33:0/539630619 >> v1:172.21.15.33:6789/0 conn(0x7efe3815f960 legacy=0x7efe3815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe377fe640 1 -- 172.21.15.33:0/539630619 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380cf0f0 con 0x7efe38145660 2024-04-25T01:09:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.772+0000 7efe367fc640 1 -- 172.21.15.33:0/539630619 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1560853165 0 0) 0x7efe380cee10 con 0x7efe3815f960 2024-04-25T01:09:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe377fe640 1 --2- 172.21.15.33:0/539630619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe38145660 0x7efe3806eb60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efe38068b80 tx=0x7efe2001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df307fa3d55946d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe367fc640 1 -- 172.21.15.33:0/539630619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe200b0070 con 0x7efe38145660 2024-04-25T01:09:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe367fc640 1 -- 172.21.15.33:0/539630619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe200ab600 con 0x7efe38145660 2024-04-25T01:09:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe367fc640 1 -- 172.21.15.33:0/539630619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe200ab9d0 con 0x7efe38145660 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- 172.21.15.33:0/539630619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe38145660 msgr2=0x7efe3806eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 --2- 172.21.15.33:0/539630619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe38145660 0x7efe3806eb60 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efe38068b80 tx=0x7efe2001c480 comp rx=0 tx=0).stop 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- 172.21.15.33:0/539630619 shutdown_connections 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 --2- 172.21.15.33:0/539630619 >> v2:172.21.15.138:3300/0 conn(0x7efe381604b0 0x7efe38071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 --2- 172.21.15.33:0/539630619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe38145660 0x7efe3806eb60 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- 172.21.15.33:0/539630619 >> 172.21.15.33:0/539630619 conn(0x7efe380b1810 msgr2=0x7efe380b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- 172.21.15.33:0/539630619 shutdown_connections 2024-04-25T01:09:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- 172.21.15.33:0/539630619 wait complete. 2024-04-25T01:09:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 Processor -- start 2024-04-25T01:09:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- start start 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe38145660 0x7efe3815b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3815f960 0x7efe380ae460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe381604b0 0x7efe38139a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efe38072290 con 0x7efe381604b0 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe38072110 con 0x7efe3815f960 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.776+0000 7efe3deb6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7efe38072410 con 0x7efe38145660 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe381604b0 0x7efe38139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe381604b0 0x7efe38139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53872/0 (socket says 172.21.15.33:53872) 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 -- 172.21.15.33:0/2581636737 learned_addr learned my addr 172.21.15.33:0/2581636737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe36ffd640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3815f960 0x7efe380ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe377fe640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe38145660 0x7efe3815b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 -- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe38145660 msgr2=0x7efe3815b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe38145660 0x7efe3815b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 -- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3815f960 msgr2=0x7efe380ae460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3815f960 0x7efe380ae460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 -- 172.21.15.33:0/2581636737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe381444a0 con 0x7efe381604b0 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe37fff640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe381604b0 0x7efe38139a10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efe3000d590 tx=0x7efe3000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe2ffff640 1 -- 172.21.15.33:0/2581636737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe300125f0 con 0x7efe381604b0 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe2ffff640 1 -- 172.21.15.33:0/2581636737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe30019070 con 0x7efe381604b0 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe38144780 con 0x7efe381604b0 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe2ffff640 1 -- 172.21.15.33:0/2581636737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe300129f0 con 0x7efe381604b0 2024-04-25T01:09:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe38144d40 con 0x7efe381604b0 2024-04-25T01:09:46.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe38069040 con 0x7efe381604b0 2024-04-25T01:09:46.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe2ffff640 1 -- 172.21.15.33:0/2581636737 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efe30012b90 con 0x7efe381604b0 2024-04-25T01:09:46.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe2ffff640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efe1c0678c0 0x7efe1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:46.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe2ffff640 1 -- 172.21.15.33:0/2581636737 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7efe300860b0 con 0x7efe381604b0 2024-04-25T01:09:46.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.780+0000 7efe377fe640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efe1c0678c0 0x7efe1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:46.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.784+0000 7efe377fe640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efe1c0678c0 0x7efe1c069d80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efe200ad000 tx=0x7efe20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.784+0000 7efe2ffff640 1 -- 172.21.15.33:0/2581636737 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7efe30056400 con 0x7efe381604b0 2024-04-25T01:09:46.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:46.992+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7efe38072fa0 con 0x7efe381604b0 2024-04-25T01:09:47.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.948+0000 7efe2ffff640 1 -- 172.21.15.33:0/2581636737 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7efe3005a700 con 0x7efe381604b0 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efe1c0678c0 msgr2=0x7efe1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efe1c0678c0 0x7efe1c069d80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efe200ad000 tx=0x7efe20008040 comp rx=0 tx=0).stop 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe381604b0 msgr2=0x7efe38139a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe381604b0 0x7efe38139a10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efe3000d590 tx=0x7efe3000da50 comp rx=0 tx=0).stop 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 shutdown_connections 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7efe1c0678c0 0x7efe1c069d80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe381604b0 0x7efe38139a10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3815f960 0x7efe380ae460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 --2- 172.21.15.33:0/2581636737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7efe38145660 0x7efe3815b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:47.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 >> 172.21.15.33:0/2581636737 conn(0x7efe380b1810 msgr2=0x7efe3806d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:47.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 shutdown_connections 2024-04-25T01:09:47.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:47.952+0000 7efe3deb6640 1 -- 172.21.15.33:0/2581636737 wait complete. 2024-04-25T01:09:47.965 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T01:09:47.965 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T01:09:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.100+0000 7f11dbdf2640 1 Processor -- start 2024-04-25T01:09:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11dbdf2640 1 -- start start 2024-04-25T01:09:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11dbdf2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 0x7f11d40cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11dbdf2640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f11d40d2ac0 0x7f11d4167cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11dbdf2640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11d40b6cc0 con 0x7f11d40d1f60 2024-04-25T01:09:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11dbdf2640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f11d40b7580 con 0x7f11d40d2ac0 2024-04-25T01:09:48.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11dbdf2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11d40b6f30 con 0x7f11d415e0d0 2024-04-25T01:09:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d938d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f11d40d1f60 0x7f11d40d23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:39880/0 (socket says 172.21.15.33:39880) 2024-04-25T01:09:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d938d640 1 -- 172.21.15.33:0/129119737 learned_addr learned my addr 172.21.15.33:0/129119737 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11da38f640 1 --2- 172.21.15.33:0/129119737 >> v2:172.21.15.138:3300/0 conn(0x7f11d40d2ac0 0x7f11d4167cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 0x7f11d40cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006588918 0 0) 0x7f11d40b6cc0 con 0x7f11d40d1f60 2024-04-25T01:09:48.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11c0003530 con 0x7f11d40d1f60 2024-04-25T01:09:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 800439305 0 0) 0x7f11c0003530 con 0x7f11d40d1f60 2024-04-25T01:09:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 msgr2=0x7f11d40cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 0x7f11d40cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 >> v2:172.21.15.138:3300/0 conn(0x7f11d40d2ac0 msgr2=0x7f11d4167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 --2- 172.21.15.33:0/129119737 >> v2:172.21.15.138:3300/0 conn(0x7f11d40d2ac0 0x7f11d4167cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d40b7210 con 0x7f11d40d1f60 2024-04-25T01:09:48.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f11c4001680 con 0x7f11d40d1f60 2024-04-25T01:09:48.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 >> v1:172.21.15.33:6789/0 conn(0x7f11d40d1f60 legacy=0x7f11d40d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:48.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c00039c0 0x7f11c0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 0x7f11c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11c0004d50 0x7f11c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11d40b7580 con 0x7f11c00039c0 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11d40b6f30 con 0x7f11c0004d50 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11c000b9d0 con 0x7f11d415e0d0 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c00039c0 0x7f11c0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d938d640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 0x7f11c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11da38f640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11c0004d50 0x7f11c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d9b8e640 1 -- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 msgr2=0x7f11c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:48.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d415e0d0 0x7f11c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d9b8e640 1 -- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11c0004d50 msgr2=0x7f11c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11c0004d50 0x7f11c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11d9b8e640 1 -- 172.21.15.33:0/129119737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c0003530 con 0x7f11c00039c0 2024-04-25T01:09:48.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.104+0000 7f11da38f640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11c0004d50 0x7f11c00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c00039c0 0x7f11c0003de0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009ac0 tx=0x7f11d001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6175362b26cdda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d0009690 con 0x7f11c00039c0 2024-04-25T01:09:48.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d00a64e0 con 0x7f11c00039c0 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11d8b8c640 1 -- 172.21.15.33:0/129119737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00a68b0 con 0x7f11c00039c0 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- 172.21.15.33:0/129119737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c00039c0 msgr2=0x7f11c0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 --2- 172.21.15.33:0/129119737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11c00039c0 0x7f11c0003de0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f11d0009ac0 tx=0x7f11d001ca50 comp rx=0 tx=0).stop 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11da38f640 1 -- 172.21.15.33:0/129119737 reap_dead start 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- 172.21.15.33:0/129119737 shutdown_connections 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- 172.21.15.33:0/129119737 >> 172.21.15.33:0/129119737 conn(0x7f11d40d3dd0 msgr2=0x7f11d40d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- 172.21.15.33:0/129119737 shutdown_connections 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- 172.21.15.33:0/129119737 wait complete. 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 Processor -- start 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- start start 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4150bb0 0x7f11d40c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d4155f30 0x7f11d40c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d40cbae0 0x7f11d40cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11d41685c0 con 0x7f11d4150bb0 2024-04-25T01:09:48.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f11d4168440 con 0x7f11d4155f30 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11dbdf2640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f11d4168740 con 0x7f11d40cbae0 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.108+0000 7f11d9b8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4150bb0 0x7f11d40c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4150bb0 0x7f11d40c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53904/0 (socket says 172.21.15.33:53904) 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 -- 172.21.15.33:0/3649855363 learned_addr learned my addr 172.21.15.33:0/3649855363 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d938d640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d4155f30 0x7f11d40c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11da38f640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d40cbae0 0x7f11d40cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 -- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d40cbae0 msgr2=0x7f11d40cbf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d40cbae0 0x7f11d40cbf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 -- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d4155f30 msgr2=0x7f11d40c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d4155f30 0x7f11d40c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 -- 172.21.15.33:0/3649855363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d40cc4b0 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11da38f640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d40cbae0 0x7f11d40cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d9b8e640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4150bb0 0x7f11d40c9900 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f11d001c880 tx=0x7f11c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11ca7fc640 1 -- 172.21.15.33:0/3649855363 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00a6890 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11ca7fc640 1 -- 172.21.15.33:0/3649855363 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11d00bf070 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d4154e90 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11ca7fc640 1 -- 172.21.15.33:0/3649855363 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d00a6c50 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d4155450 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d4069040 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11ca7fc640 1 -- 172.21.15.33:0/3649855363 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f11d0005fa0 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11ca7fc640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11b80678c0 0x7f11b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11ca7fc640 1 -- 172.21.15.33:0/3649855363 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f11d0120f80 con 0x7f11d4150bb0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.112+0000 7f11d938d640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11b80678c0 0x7f11b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.116+0000 7f11d938d640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11b80678c0 0x7f11b8069d80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11d40d11e0 tx=0x7f11c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.116+0000 7f11ca7fc640 1 -- 172.21.15.33:0/3649855363 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f11d0016760 con 0x7f11d4150bb0 2024-04-25T01:09:48.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.312+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f11d40c8690 con 0x7f11d4150bb0 2024-04-25T01:09:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.956+0000 7f11ca7fc640 1 -- 172.21.15.33:0/3649855363 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f11d00f1980 con 0x7f11d4150bb0 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11b80678c0 msgr2=0x7f11b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11b80678c0 0x7f11b8069d80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f11d40d11e0 tx=0x7f11c4009040 comp rx=0 tx=0).stop 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4150bb0 msgr2=0x7f11d40c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4150bb0 0x7f11d40c9900 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f11d001c880 tx=0x7f11c0005840 comp rx=0 tx=0).stop 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 shutdown_connections 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f11b80678c0 0x7f11b8069d80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f11d40cbae0 0x7f11d40cbf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11d4155f30 0x7f11d40c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 --2- 172.21.15.33:0/3649855363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11d4150bb0 0x7f11d40c9900 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 >> 172.21.15.33:0/3649855363 conn(0x7f11d40d3dd0 msgr2=0x7f11d40b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 shutdown_connections 2024-04-25T01:09:48.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:48.960+0000 7f11dbdf2640 1 -- 172.21.15.33:0/3649855363 wait complete. 2024-04-25T01:09:48.976 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T01:09:48.976 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:09:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fc609e640 1 Processor -- start 2024-04-25T01:09:49.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fc609e640 1 -- start start 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fc609e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fc0161860 0x7f2fc00d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fc609e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc00cfd70 con 0x7f2fc00cf8d0 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fc609e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc00d1370 con 0x7f2fc00d49d0 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fc609e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc00d2ad0 con 0x7f2fc0161860 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fbf7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fc0161860 0x7f2fc00d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fbf7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f2fc0161860 0x7f2fc00d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42966/0 (socket says 172.21.15.33:42966) 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fbf7fe640 1 -- 172.21.15.33:0/1072356379 learned_addr learned my addr 172.21.15.33:0/1072356379 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.112+0000 7f2fbf7fe640 1 -- 172.21.15.33:0/1072356379 >> v1:172.21.15.138:6789/0 conn(0x7f2fc00d49d0 legacy=0x7f2fc00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fbf7fe640 1 -- 172.21.15.33:0/1072356379 >> v1:172.21.15.33:6789/0 conn(0x7f2fc00cf8d0 legacy=0x7f2fc00b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fbf7fe640 1 -- 172.21.15.33:0/1072356379 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc00d2760 con 0x7f2fc0161860 2024-04-25T01:09:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fbf7fe640 1 --2- 172.21.15.33:0/1072356379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc0161860 0x7f2fc00d01c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009870 tx=0x7f2fb001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2e3af6618a95f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fbe7fc640 1 -- 172.21.15.33:0/1072356379 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00aa620 con 0x7f2fc0161860 2024-04-25T01:09:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fbe7fc640 1 -- 172.21.15.33:0/1072356379 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb00af070 con 0x7f2fc0161860 2024-04-25T01:09:49.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fbe7fc640 1 -- 172.21.15.33:0/1072356379 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00aa9e0 con 0x7f2fc0161860 2024-04-25T01:09:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fc609e640 1 -- 172.21.15.33:0/1072356379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc0161860 msgr2=0x7f2fc00d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fc609e640 1 --2- 172.21.15.33:0/1072356379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc0161860 0x7f2fc00d01c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009870 tx=0x7f2fb001c520 comp rx=0 tx=0).stop 2024-04-25T01:09:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fc609e640 1 -- 172.21.15.33:0/1072356379 shutdown_connections 2024-04-25T01:09:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fc609e640 1 --2- 172.21.15.33:0/1072356379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc0161860 0x7f2fc00d01c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fc609e640 1 -- 172.21.15.33:0/1072356379 >> 172.21.15.33:0/1072356379 conn(0x7f2fc00d3db0 msgr2=0x7f2fc00d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fc609e640 1 -- 172.21.15.33:0/1072356379 shutdown_connections 2024-04-25T01:09:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.116+0000 7f2fc609e640 1 -- 172.21.15.33:0/1072356379 wait complete. 2024-04-25T01:09:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 Processor -- start 2024-04-25T01:09:49.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 -- start start 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fc00cf8d0 0x7f2fc006ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00d49d0 0x7f2fc01562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fc0161860 0x7f2fc014e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00b4a30 con 0x7f2fc00d49d0 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc00b48b0 con 0x7f2fc00cf8d0 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc016bf70 con 0x7f2fc0161860 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00d49d0 0x7f2fc01562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00d49d0 0x7f2fc01562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42978/0 (socket says 172.21.15.33:42978) 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 -- 172.21.15.33:0/3090926857 learned_addr learned my addr 172.21.15.33:0/3090926857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbf7fe640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fc00cf8d0 0x7f2fc006ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 -- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fc0161860 msgr2=0x7f2fc014e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbffff640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fc0161860 0x7f2fc014e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fc0161860 0x7f2fc014e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 -- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fc00cf8d0 msgr2=0x7f2fc006ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fc00cf8d0 0x7f2fc006ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 -- 172.21.15.33:0/3090926857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc0158070 con 0x7f2fc00d49d0 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbf7fe640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fc00cf8d0 0x7f2fc006ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fbeffd640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00d49d0 0x7f2fc01562c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800ba50 tx=0x7f2fa800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fb7fff640 1 -- 172.21.15.33:0/3090926857 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8019070 con 0x7f2fc00d49d0 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fb7fff640 1 -- 172.21.15.33:0/3090926857 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa800cc20 con 0x7f2fc00d49d0 2024-04-25T01:09:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc0158350 con 0x7f2fc00d49d0 2024-04-25T01:09:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fb7fff640 1 -- 172.21.15.33:0/3090926857 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8014680 con 0x7f2fc00d49d0 2024-04-25T01:09:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc0158910 con 0x7f2fc00d49d0 2024-04-25T01:09:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fc0069040 con 0x7f2fc00d49d0 2024-04-25T01:09:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fb7fff640 1 -- 172.21.15.33:0/3090926857 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2fa801c070 con 0x7f2fc00d49d0 2024-04-25T01:09:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.120+0000 7f2fb7fff640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2fa00678c0 0x7f2fa0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.124+0000 7f2fb7fff640 1 -- 172.21.15.33:0/3090926857 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2fa8085a80 con 0x7f2fc00d49d0 2024-04-25T01:09:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.124+0000 7f2fbf7fe640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2fa00678c0 0x7f2fa0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.124+0000 7f2fbf7fe640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2fa00678c0 0x7f2fa0069d80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00ae000 tx=0x7f2fb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.128+0000 7f2fb7fff640 1 -- 172.21.15.33:0/3090926857 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2fa8055e70 con 0x7f2fc00d49d0 2024-04-25T01:09:49.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.320+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fc01570b0 con 0x7f2fc00d49d0 2024-04-25T01:09:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fb7fff640 1 -- 172.21.15.33:0/3090926857 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f2fa805a170 con 0x7f2fc00d49d0 2024-04-25T01:09:49.315 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:49.315 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3228956110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3228956110},{"type":"v1","addr":"172.21.15.138:6835","nonce":3228956110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2116493369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2116493369},{"type":"v1","addr":"172.21.15.33:6835","nonce":2116493369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2610873006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2610873006},{"type":"v1","addr":"172.21.15.138:6837","nonce":2610873006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:09:45.288708+0000","modified":"2024-04-25T01:09:48.318253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/3726672082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3726672082},{"type":"v1","addr":"172.21.15.33:6834","nonce":3726672082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-25T01:09:49.315 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-25T01:09:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2fa00678c0 msgr2=0x7f2fa0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2fa00678c0 0x7f2fa0069d80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00ae000 tx=0x7f2fb0008040 comp rx=0 tx=0).stop 2024-04-25T01:09:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00d49d0 msgr2=0x7f2fc01562c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00d49d0 0x7f2fc01562c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800ba50 tx=0x7f2fa800bf10 comp rx=0 tx=0).stop 2024-04-25T01:09:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 shutdown_connections 2024-04-25T01:09:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2fa00678c0 0x7f2fa0069d80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2fc0161860 0x7f2fc014e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2fc00d49d0 0x7f2fc01562c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 --2- 172.21.15.33:0/3090926857 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fc00cf8d0 0x7f2fc006ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 >> 172.21.15.33:0/3090926857 conn(0x7f2fc00d3db0 msgr2=0x7f2fc00b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 shutdown_connections 2024-04-25T01:09:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.324+0000 7f2fc609e640 1 -- 172.21.15.33:0/3090926857 wait complete. 2024-04-25T01:09:49.334 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:09:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680f74a640 1 Processor -- start 2024-04-25T01:09:49.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680f74a640 1 -- start start 2024-04-25T01:09:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680f74a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6808161690 0x7f68080d49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680f74a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6808161a70 0x7f68080d0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680f74a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68080b49a0 con 0x7f68080b2d30 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680f74a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f68080b4c80 con 0x7f6808161a70 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680f74a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f68080b4f60 con 0x7f6808161690 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680cce5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6808161a70 0x7f68080d0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680cce5640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6808161a70 0x7f68080d0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42992/0 (socket says 172.21.15.33:42992) 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680cce5640 1 -- 172.21.15.33:0/2168452415 learned_addr learned my addr 172.21.15.33:0/2168452415 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680d4e6640 1 --2- 172.21.15.33:0/2168452415 >> v2:172.21.15.138:3300/0 conn(0x7f6808161690 0x7f68080d49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680cce5640 1 -- 172.21.15.33:0/2168452415 >> v2:172.21.15.138:3300/0 conn(0x7f6808161690 msgr2=0x7f68080d49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680cce5640 1 --2- 172.21.15.33:0/2168452415 >> v2:172.21.15.138:3300/0 conn(0x7f6808161690 0x7f68080d49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680cce5640 1 -- 172.21.15.33:0/2168452415 >> v1:172.21.15.33:6789/0 conn(0x7f68080b2d30 legacy=0x7f68080b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f67fffff640 1 -- 172.21.15.33:0/2168452415 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1592781280 0 0) 0x7f68080b49a0 con 0x7f68080b2d30 2024-04-25T01:09:49.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.464+0000 7f680cce5640 1 -- 172.21.15.33:0/2168452415 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68080b5240 con 0x7f6808161a70 2024-04-25T01:09:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680cce5640 1 --2- 172.21.15.33:0/2168452415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f68080d0820 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f68080b6910 tx=0x7f67f801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffef96759543b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f67fffff640 1 -- 172.21.15.33:0/2168452415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80ab5f0 con 0x7f6808161a70 2024-04-25T01:09:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f67fffff640 1 -- 172.21.15.33:0/2168452415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67f80b0070 con 0x7f6808161a70 2024-04-25T01:09:49.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f67fffff640 1 -- 172.21.15.33:0/2168452415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f80ab9b0 con 0x7f6808161a70 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- 172.21.15.33:0/2168452415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 msgr2=0x7f68080d0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 --2- 172.21.15.33:0/2168452415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f68080d0820 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f68080b6910 tx=0x7f67f801c3d0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- 172.21.15.33:0/2168452415 shutdown_connections 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 --2- 172.21.15.33:0/2168452415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f68080d0820 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 --2- 172.21.15.33:0/2168452415 >> v2:172.21.15.138:3300/0 conn(0x7f6808161690 0x7f68080d49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- 172.21.15.33:0/2168452415 >> 172.21.15.33:0/2168452415 conn(0x7f680809d100 msgr2=0x7f68080d3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- 172.21.15.33:0/2168452415 shutdown_connections 2024-04-25T01:09:49.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- 172.21.15.33:0/2168452415 wait complete. 2024-04-25T01:09:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 Processor -- start 2024-04-25T01:09:49.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- start start 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68080b2d30 0x7f68080c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6808161690 0x7f68080ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f6808158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f680816af50 con 0x7f6808161a70 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f68080b46d0 con 0x7f68080b2d30 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.468+0000 7f680f74a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f680816b0d0 con 0x7f6808161690 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f6808158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f6808158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43008/0 (socket says 172.21.15.33:43008) 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 -- 172.21.15.33:0/310568408 learned_addr learned my addr 172.21.15.33:0/310568408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680cce5640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6808161690 0x7f68080ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680d4e6640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68080b2d30 0x7f68080c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 -- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6808161690 msgr2=0x7f68080ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6808161690 0x7f68080ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 -- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68080b2d30 msgr2=0x7f68080c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68080b2d30 0x7f68080c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 -- 172.21.15.33:0/310568408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68080c7d50 con 0x7f6808161a70 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680cce5640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6808161690 0x7f68080ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680d4e6640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68080b2d30 0x7f68080c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680dce7640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f6808158250 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f680000c780 tx=0x7f680000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f67fdffb640 1 -- 172.21.15.33:0/310568408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800003db0 con 0x7f6808161a70 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f67fdffb640 1 -- 172.21.15.33:0/310568408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6800005070 con 0x7f6808161a70 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68080c8030 con 0x7f6808161a70 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f67fdffb640 1 -- 172.21.15.33:0/310568408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800005460 con 0x7f6808161a70 2024-04-25T01:09:49.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68080c85f0 con 0x7f6808161a70 2024-04-25T01:09:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6808069040 con 0x7f6808161a70 2024-04-25T01:09:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f67fdffb640 1 -- 172.21.15.33:0/310568408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6800005600 con 0x7f6808161a70 2024-04-25T01:09:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f67fdffb640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67ec0678c0 0x7f67ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f67fdffb640 1 -- 172.21.15.33:0/310568408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6800086910 con 0x7f6808161a70 2024-04-25T01:09:49.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680d4e6640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67ec0678c0 0x7f67ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.472+0000 7f680d4e6640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67ec0678c0 0x7f67ec069d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6804009840 tx=0x7f6804008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.476+0000 7f67fdffb640 1 -- 172.21.15.33:0/310568408 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f68000572a0 con 0x7f6808161a70 2024-04-25T01:09:49.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.640+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68080b0d40 con 0x7f6808161a70 2024-04-25T01:09:49.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f67fdffb640 1 -- 172.21.15.33:0/310568408 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16930 (secure 0 0 0) 0x7f6800056ba0 con 0x7f6808161a70 2024-04-25T01:09:49.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:49.635 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:09:46.936238+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:09:43.725950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T01:09:44.271067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:09:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67ec0678c0 msgr2=0x7f67ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67ec0678c0 0x7f67ec069d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6804009840 tx=0x7f6804008040 comp rx=0 tx=0).stop 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 msgr2=0x7f6808158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f6808158250 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f680000c780 tx=0x7f680000cc40 comp rx=0 tx=0).stop 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 shutdown_connections 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67ec0678c0 0x7f67ec069d80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6808161a70 0x7f6808158250 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6808161690 0x7f68080ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 --2- 172.21.15.33:0/310568408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68080b2d30 0x7f68080c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 >> 172.21.15.33:0/310568408 conn(0x7f680809d100 msgr2=0x7f68080b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 shutdown_connections 2024-04-25T01:09:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.644+0000 7f680f74a640 1 -- 172.21.15.33:0/310568408 wait complete. 2024-04-25T01:09:49.655 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T01:09:49.655 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:09:49.723 INFO:tasks.ceph.mds.c.smithi033.stderr:2024-04-25T01:09:49.732+0000 7fc145c08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:09:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f73c30640 1 Processor -- start 2024-04-25T01:09:49.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f73c30640 1 -- start start 2024-04-25T01:09:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f73c30640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f2f6c0d48d0 0x7f2f6c0cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f73c30640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c0d10b0 con 0x7f2f6c0d03e0 2024-04-25T01:09:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f73c30640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f6c0d1390 con 0x7f2f6c15e470 2024-04-25T01:09:49.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f73c30640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f2f6c0d1de0 con 0x7f2f6c0d48d0 2024-04-25T01:09:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f2f6c0d03e0 0x7f2f6c0d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53496/0 (socket says 172.21.15.33:53496) 2024-04-25T01:09:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 -- 172.21.15.33:0/3488173468 learned_addr learned my addr 172.21.15.33:0/3488173468 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f721cd640 1 --2- 172.21.15.33:0/3488173468 >> v2:172.21.15.138:3300/0 conn(0x7f2f6c0d48d0 0x7f2f6c0cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1306792824 0 0) 0x7f2f6c0d10b0 con 0x7f2f6c0d03e0 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f54003530 con 0x7f2f6c0d03e0 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2863026140 0 0) 0x7f2f54003530 con 0x7f2f6c0d03e0 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 >> v2:172.21.15.138:3300/0 conn(0x7f2f6c0d48d0 msgr2=0x7f2f6c0cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 --2- 172.21.15.33:0/3488173468 >> v2:172.21.15.138:3300/0 conn(0x7f2f6c0d48d0 0x7f2f6c0cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 >> v1:172.21.15.138:6789/0 conn(0x7f2f6c15e470 legacy=0x7f2f6c0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c0d1670 con 0x7f2f6c0d03e0 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f2f5c001680 con 0x7f2f6c0d03e0 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 >> v1:172.21.15.33:6789/0 conn(0x7f2f6c0d03e0 legacy=0x7f2f6c0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f540039c0 0x7f2f54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f54004810 0x7f2f54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540051d0 0x7f2f5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0d1de0 con 0x7f2f540039c0 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5400c170 con 0x7f2f54004810 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5400c370 con 0x7f2f540051d0 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f540039c0 0x7f2f54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f711cb640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f54004810 0x7f2f54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f721cd640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540051d0 0x7f2f5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 -- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540051d0 msgr2=0x7f2f5400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540051d0 0x7f2f5400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 -- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f54004810 msgr2=0x7f2f54004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f54004810 0x7f2f54004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f719cc640 1 -- 172.21.15.33:0/3488173468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f54003530 con 0x7f2f540039c0 2024-04-25T01:09:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.784+0000 7f2f721cd640 1 -- 172.21.15.33:0/3488173468 reap_dead start 2024-04-25T01:09:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f719cc640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f540039c0 0x7f2f54003de0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c290 tx=0x7f2f5c01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c6c69ced1b7f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c007d40 con 0x7f2f540039c0 2024-04-25T01:09:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f5c0a9020 con 0x7f2f540039c0 2024-04-25T01:09:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f709ca640 1 -- 172.21.15.33:0/3488173468 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c005ae0 con 0x7f2f540039c0 2024-04-25T01:09:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 -- 172.21.15.33:0/3488173468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f540039c0 msgr2=0x7f2f54003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f540039c0 0x7f2f54003de0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00c290 tx=0x7f2f5c01cb80 comp rx=0 tx=0).stop 2024-04-25T01:09:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 -- 172.21.15.33:0/3488173468 shutdown_connections 2024-04-25T01:09:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 --2- 172.21.15.33:0/3488173468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f540039c0 0x7f2f54003de0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 -- 172.21.15.33:0/3488173468 >> 172.21.15.33:0/3488173468 conn(0x7f2f6c0d3db0 msgr2=0x7f2f6c0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 -- 172.21.15.33:0/3488173468 shutdown_connections 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 -- 172.21.15.33:0/3488173468 wait complete. 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 Processor -- start 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.788+0000 7f2f73c30640 1 -- start start 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540039c0 0x7f2f6c150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f6c0b6920 0x7f2f6c0b6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b8f60 0x7f2f6c0b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0d2e10 con 0x7f2f6c0b6920 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6c0d2c90 con 0x7f2f6c0b8f60 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6c0d2f90 con 0x7f2f540039c0 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f6c0b6920 0x7f2f6c0b6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f6c0b6920 0x7f2f6c0b6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43026/0 (socket says 172.21.15.33:43026) 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 -- 172.21.15.33:0/4277693135 learned_addr learned my addr 172.21.15.33:0/4277693135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f719cc640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540039c0 0x7f2f6c150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f721cd640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b8f60 0x7f2f6c0b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 -- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540039c0 msgr2=0x7f2f6c150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540039c0 0x7f2f6c150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 -- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b8f60 msgr2=0x7f2f6c0b93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b8f60 0x7f2f6c0b93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 -- 172.21.15.33:0/4277693135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6c150740 con 0x7f2f6c0b6920 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f719cc640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540039c0 0x7f2f6c150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f711cb640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f6c0b6920 0x7f2f6c0b6da0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e820 tx=0x7f2f6000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f6a7fc640 1 -- 172.21.15.33:0/4277693135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60003b90 con 0x7f2f6c0b6920 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f6a7fc640 1 -- 172.21.15.33:0/4277693135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f60018070 con 0x7f2f6c0b6920 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6c0b73e0 con 0x7f2f6c0b6920 2024-04-25T01:09:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f6a7fc640 1 -- 172.21.15.33:0/4277693135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f600050d0 con 0x7f2f6c0b6920 2024-04-25T01:09:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6c0b79a0 con 0x7f2f6c0b6920 2024-04-25T01:09:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6c069040 con 0x7f2f6c0b6920 2024-04-25T01:09:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f6a7fc640 1 -- 172.21.15.33:0/4277693135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2f60005270 con 0x7f2f6c0b6920 2024-04-25T01:09:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f6a7fc640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f580678c0 0x7f2f58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.792+0000 7f2f6a7fc640 1 -- 172.21.15.33:0/4277693135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2f60085ce0 con 0x7f2f6c0b6920 2024-04-25T01:09:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.796+0000 7f2f719cc640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f580678c0 0x7f2f58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.796+0000 7f2f719cc640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f580678c0 0x7f2f58069d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0aa000 tx=0x7f2f5c001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.800+0000 7f2f6a7fc640 1 -- 172.21.15.33:0/4277693135 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f2f60056720 con 0x7f2f6c0b6920 2024-04-25T01:09:49.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.952+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2f6c160550 con 0x7f2f580678c0 2024-04-25T01:09:49.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.980+0000 7f2f6a7fc640 1 -- 172.21.15.33:0/4277693135 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7f2f6c160550 con 0x7f2f580678c0 2024-04-25T01:09:49.977 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:49.979 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:09:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f580678c0 msgr2=0x7f2f58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f580678c0 0x7f2f58069d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c0aa000 tx=0x7f2f5c001a50 comp rx=0 tx=0).stop 2024-04-25T01:09:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f6c0b6920 msgr2=0x7f2f6c0b6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f6c0b6920 0x7f2f6c0b6da0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e820 tx=0x7f2f6000ece0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 shutdown_connections 2024-04-25T01:09:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f2f580678c0 0x7f2f58069d80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f6c0b8f60 0x7f2f6c0b93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f6c0b6920 0x7f2f6c0b6da0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 --2- 172.21.15.33:0/4277693135 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f2f540039c0 0x7f2f6c150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 >> 172.21.15.33:0/4277693135 conn(0x7f2f6c0d3db0 msgr2=0x7f2f6c160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:49.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 shutdown_connections 2024-04-25T01:09:49.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:49.984+0000 7f2f73c30640 1 -- 172.21.15.33:0/4277693135 wait complete. 2024-04-25T01:09:49.993 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-04-25T01:09:49.118329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229108,"kb_used_data":4596,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745612,"statfs":{"total":773094113280,"available":772859506688,"internally_reserved":0,"allocated":4706304,"data_stored":2682188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.218252"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949170+0000","last_change":"2024-04-25T01:09:45.956517+0000","last_active":"2024-04-25T01:09:46.949170+0000","last_peered":"2024-04-25T01:09:46.949170+0000","last_clean":"2024-04-25T01:09:46.949170+0000","last_became_active":"2024-04-25T01:09:45.956210+0000","last_became_peered":"2024-04-25T01:09:45.956210+0000","last_unstale":"2024-04-25T01:09:46.949170+0000","last_undegraded":"2024-04-25T01:09:46.949170+0000","last_fullsized":"2024-04-25T01:09:46.949170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:30.247260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948621+0000","last_change":"2024-04-25T01:09:45.951511+0000","last_active":"2024-04-25T01:09:46.948621+0000","last_peered":"2024-04-25T01:09:46.948621+0000","last_clean":"2024-04-25T01:09:46.948621+0000","last_became_active":"2024-04-25T01:09:45.950919+0000","last_became_peered":"2024-04-25T01:09:45.950919+0000","last_unstale":"2024-04-25T01:09:46.948621+0000","last_undegraded":"2024-04-25T01:09:46.948621+0000","last_fullsized":"2024-04-25T01:09:46.948621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:04.759348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991307+0000","last_change":"2024-04-25T01:09:45.989625+0000","last_active":"2024-04-25T01:09:45.991307+0000","last_peered":"2024-04-25T01:09:45.991307+0000","last_clean":"2024-04-25T01:09:45.991307+0000","last_became_active":"2024-04-25T01:09:45.989290+0000","last_became_peered":"2024-04-25T01:09:45.989290+0000","last_unstale":"2024-04-25T01:09:45.991307+0000","last_undegraded":"2024-04-25T01:09:45.991307+0000","last_fullsized":"2024-04-25T01:09:45.991307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:17.887483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958005+0000","last_change":"2024-04-25T01:09:45.985227+0000","last_active":"2024-04-25T01:09:46.958005+0000","last_peered":"2024-04-25T01:09:46.958005+0000","last_clean":"2024-04-25T01:09:46.958005+0000","last_became_active":"2024-04-25T01:09:45.984936+0000","last_became_peered":"2024-04-25T01:09:45.984936+0000","last_unstale":"2024-04-25T01:09:46.958005+0000","last_undegraded":"2024-04-25T01:09:46.958005+0000","last_fullsized":"2024-04-25T01:09:46.958005+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:05.022909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958036+0000","last_change":"2024-04-25T01:09:45.974764+0000","last_active":"2024-04-25T01:09:46.958036+0000","last_peered":"2024-04-25T01:09:46.958036+0000","last_clean":"2024-04-25T01:09:46.958036+0000","last_became_active":"2024-04-25T01:09:45.974517+0000","last_became_peered":"2024-04-25T01:09:45.974517+0000","last_unstale":"2024-04-25T01:09:46.958036+0000","last_undegraded":"2024-04-25T01:09:46.958036+0000","last_fullsized":"2024-04-25T01:09:46.958036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:29.422454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.955016+0000","last_change":"2024-04-25T01:09:45.952003+0000","last_active":"2024-04-25T01:09:45.955016+0000","last_peered":"2024-04-25T01:09:45.955016+0000","last_clean":"2024-04-25T01:09:45.955016+0000","last_became_active":"2024-04-25T01:09:45.951686+0000","last_became_peered":"2024-04-25T01:09:45.951686+0000","last_unstale":"2024-04-25T01:09:45.955016+0000","last_undegraded":"2024-04-25T01:09:45.955016+0000","last_fullsized":"2024-04-25T01:09:45.955016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:56.615296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989992+0000","last_change":"2024-04-25T01:09:45.976515+0000","last_active":"2024-04-25T01:09:45.989992+0000","last_peered":"2024-04-25T01:09:45.989992+0000","last_clean":"2024-04-25T01:09:45.989992+0000","last_became_active":"2024-04-25T01:09:45.960237+0000","last_became_peered":"2024-04-25T01:09:45.960237+0000","last_unstale":"2024-04-25T01:09:45.989992+0000","last_undegraded":"2024-04-25T01:09:45.989992+0000","last_fullsized":"2024-04-25T01:09:45.989992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:38.105948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.947796+0000","last_change":"2024-04-25T01:09:45.984500+0000","last_active":"2024-04-25T01:09:46.947796+0000","last_peered":"2024-04-25T01:09:46.947796+0000","last_clean":"2024-04-25T01:09:46.947796+0000","last_became_active":"2024-04-25T01:09:45.984132+0000","last_became_peered":"2024-04-25T01:09:45.984132+0000","last_unstale":"2024-04-25T01:09:46.947796+0000","last_undegraded":"2024-04-25T01:09:46.947796+0000","last_fullsized":"2024-04-25T01:09:46.947796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:47.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.987523+0000","last_change":"2024-04-25T01:09:45.986604+0000","last_active":"2024-04-25T01:09:45.987523+0000","last_peered":"2024-04-25T01:09:45.987523+0000","last_clean":"2024-04-25T01:09:45.987523+0000","last_became_active":"2024-04-25T01:09:45.986334+0000","last_became_peered":"2024-04-25T01:09:45.986334+0000","last_unstale":"2024-04-25T01:09:45.987523+0000","last_undegraded":"2024-04-25T01:09:45.987523+0000","last_fullsized":"2024-04-25T01:09:45.987523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:35.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992385+0000","last_change":"2024-04-25T01:09:45.987850+0000","last_active":"2024-04-25T01:09:45.992385+0000","last_peered":"2024-04-25T01:09:45.992385+0000","last_clean":"2024-04-25T01:09:45.992385+0000","last_became_active":"2024-04-25T01:09:45.987281+0000","last_became_peered":"2024-04-25T01:09:45.987281+0000","last_unstale":"2024-04-25T01:09:45.992385+0000","last_undegraded":"2024-04-25T01:09:45.992385+0000","last_fullsized":"2024-04-25T01:09:45.992385+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:14.111400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992346+0000","last_change":"2024-04-25T01:09:45.987178+0000","last_active":"2024-04-25T01:09:45.992346+0000","last_peered":"2024-04-25T01:09:45.992346+0000","last_clean":"2024-04-25T01:09:45.992346+0000","last_became_active":"2024-04-25T01:09:45.983271+0000","last_became_peered":"2024-04-25T01:09:45.983271+0000","last_unstale":"2024-04-25T01:09:45.992346+0000","last_undegraded":"2024-04-25T01:09:45.992346+0000","last_fullsized":"2024-04-25T01:09:45.992346+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:07.230148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.953791+0000","last_change":"2024-04-25T01:09:45.949145+0000","last_active":"2024-04-25T01:09:45.953791+0000","last_peered":"2024-04-25T01:09:45.953791+0000","last_clean":"2024-04-25T01:09:45.953791+0000","last_became_active":"2024-04-25T01:09:45.948672+0000","last_became_peered":"2024-04-25T01:09:45.948672+0000","last_unstale":"2024-04-25T01:09:45.953791+0000","last_undegraded":"2024-04-25T01:09:45.953791+0000","last_fullsized":"2024-04-25T01:09:45.953791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:23.810444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989856+0000","last_change":"2024-04-25T01:09:45.977243+0000","last_active":"2024-04-25T01:09:45.989856+0000","last_peered":"2024-04-25T01:09:45.989856+0000","last_clean":"2024-04-25T01:09:45.989856+0000","last_became_active":"2024-04-25T01:09:45.976641+0000","last_became_peered":"2024-04-25T01:09:45.976641+0000","last_unstale":"2024-04-25T01:09:45.989856+0000","last_undegraded":"2024-04-25T01:09:45.989856+0000","last_fullsized":"2024-04-25T01:09:45.989856+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:51.578818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949053+0000","last_change":"2024-04-25T01:09:45.973163+0000","last_active":"2024-04-25T01:09:46.949053+0000","last_peered":"2024-04-25T01:09:46.949053+0000","last_clean":"2024-04-25T01:09:46.949053+0000","last_became_active":"2024-04-25T01:09:45.971174+0000","last_became_peered":"2024-04-25T01:09:45.971174+0000","last_unstale":"2024-04-25T01:09:46.949053+0000","last_undegraded":"2024-04-25T01:09:46.949053+0000","last_fullsized":"2024-04-25T01:09:46.949053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:34.705500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.982532+0000","last_change":"2024-04-25T01:09:45.959115+0000","last_active":"2024-04-25T01:09:45.982532+0000","last_peered":"2024-04-25T01:09:45.982532+0000","last_clean":"2024-04-25T01:09:45.982532+0000","last_became_active":"2024-04-25T01:09:45.958380+0000","last_became_peered":"2024-04-25T01:09:45.958380+0000","last_unstale":"2024-04-25T01:09:45.982532+0000","last_undegraded":"2024-04-25T01:09:45.982532+0000","last_fullsized":"2024-04-25T01:09:45.982532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:17.887483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.760930+0000","last_change":"2024-04-25T01:09:45.992991+0000","last_active":"2024-04-25T01:09:47.760930+0000","last_peered":"2024-04-25T01:09:47.760930+0000","last_clean":"2024-04-25T01:09:47.760930+0000","last_became_active":"2024-04-25T01:09:45.990811+0000","last_became_peered":"2024-04-25T01:09:45.990811+0000","last_unstale":"2024-04-25T01:09:47.760930+0000","last_undegraded":"2024-04-25T01:09:47.760930+0000","last_fullsized":"2024-04-25T01:09:47.760930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:46.283959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.762050+0000","last_change":"2024-04-25T01:09:45.987765+0000","last_active":"2024-04-25T01:09:47.762050+0000","last_peered":"2024-04-25T01:09:47.762050+0000","last_clean":"2024-04-25T01:09:47.762050+0000","last_became_active":"2024-04-25T01:09:45.987230+0000","last_became_peered":"2024-04-25T01:09:45.987230+0000","last_unstale":"2024-04-25T01:09:47.762050+0000","last_undegraded":"2024-04-25T01:09:47.762050+0000","last_fullsized":"2024-04-25T01:09:47.762050+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.976089+0000","last_change":"2024-04-25T01:09:45.974198+0000","last_active":"2024-04-25T01:09:45.976089+0000","last_peered":"2024-04-25T01:09:45.976089+0000","last_clean":"2024-04-25T01:09:45.976089+0000","last_became_active":"2024-04-25T01:09:45.973480+0000","last_became_peered":"2024-04-25T01:09:45.973480+0000","last_unstale":"2024-04-25T01:09:45.976089+0000","last_undegraded":"2024-04-25T01:09:45.976089+0000","last_fullsized":"2024-04-25T01:09:45.976089+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:43.736020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958359+0000","last_change":"2024-04-25T01:09:45.943626+0000","last_active":"2024-04-25T01:09:46.958359+0000","last_peered":"2024-04-25T01:09:46.958359+0000","last_clean":"2024-04-25T01:09:46.958359+0000","last_became_active":"2024-04-25T01:09:45.943334+0000","last_became_peered":"2024-04-25T01:09:45.943334+0000","last_unstale":"2024-04-25T01:09:46.958359+0000","last_undegraded":"2024-04-25T01:09:46.958359+0000","last_fullsized":"2024-04-25T01:09:46.958359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:35.161718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.985995+0000","last_change":"2024-04-25T01:09:45.981951+0000","last_active":"2024-04-25T01:09:45.985995+0000","last_peered":"2024-04-25T01:09:45.985995+0000","last_clean":"2024-04-25T01:09:45.985995+0000","last_became_active":"2024-04-25T01:09:45.981250+0000","last_became_peered":"2024-04-25T01:09:45.981250+0000","last_unstale":"2024-04-25T01:09:45.985995+0000","last_undegraded":"2024-04-25T01:09:45.985995+0000","last_fullsized":"2024-04-25T01:09:45.985995+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:01.017792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989925+0000","last_change":"2024-04-25T01:09:45.980355+0000","last_active":"2024-04-25T01:09:45.989925+0000","last_peered":"2024-04-25T01:09:45.989925+0000","last_clean":"2024-04-25T01:09:45.989925+0000","last_became_active":"2024-04-25T01:09:45.950989+0000","last_became_peered":"2024-04-25T01:09:45.950989+0000","last_unstale":"2024-04-25T01:09:45.989925+0000","last_undegraded":"2024-04-25T01:09:45.989925+0000","last_fullsized":"2024-04-25T01:09:45.989925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:52.298327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949972+0000","last_change":"2024-04-25T01:09:45.972529+0000","last_active":"2024-04-25T01:09:46.949972+0000","last_peered":"2024-04-25T01:09:46.949972+0000","last_clean":"2024-04-25T01:09:46.949972+0000","last_became_active":"2024-04-25T01:09:45.970396+0000","last_became_peered":"2024-04-25T01:09:45.970396+0000","last_unstale":"2024-04-25T01:09:46.949972+0000","last_undegraded":"2024-04-25T01:09:46.949972+0000","last_fullsized":"2024-04-25T01:09:46.949972+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:06.808576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761308+0000","last_change":"2024-04-25T01:09:45.962771+0000","last_active":"2024-04-25T01:09:47.761308+0000","last_peered":"2024-04-25T01:09:47.761308+0000","last_clean":"2024-04-25T01:09:47.761308+0000","last_became_active":"2024-04-25T01:09:45.952747+0000","last_became_peered":"2024-04-25T01:09:45.952747+0000","last_unstale":"2024-04-25T01:09:47.761308+0000","last_undegraded":"2024-04-25T01:09:47.761308+0000","last_fullsized":"2024-04-25T01:09:47.761308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:17.084255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.957754+0000","last_change":"2024-04-25T01:09:45.956306+0000","last_active":"2024-04-25T01:09:46.957754+0000","last_peered":"2024-04-25T01:09:46.957754+0000","last_clean":"2024-04-25T01:09:46.957754+0000","last_became_active":"2024-04-25T01:09:45.954871+0000","last_became_peered":"2024-04-25T01:09:45.954871+0000","last_unstale":"2024-04-25T01:09:46.957754+0000","last_undegraded":"2024-04-25T01:09:46.957754+0000","last_fullsized":"2024-04-25T01:09:46.957754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:01.187462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.957376+0000","last_change":"2024-04-25T01:09:45.945731+0000","last_active":"2024-04-25T01:09:45.957376+0000","last_peered":"2024-04-25T01:09:45.957376+0000","last_clean":"2024-04-25T01:09:45.957376+0000","last_became_active":"2024-04-25T01:09:45.945040+0000","last_became_peered":"2024-04-25T01:09:45.945040+0000","last_unstale":"2024-04-25T01:09:45.957376+0000","last_undegraded":"2024-04-25T01:09:45.957376+0000","last_fullsized":"2024-04-25T01:09:45.957376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:35.161718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.993249+0000","last_change":"2024-04-25T01:09:45.991405+0000","last_active":"2024-04-25T01:09:45.993249+0000","last_peered":"2024-04-25T01:09:45.993249+0000","last_clean":"2024-04-25T01:09:45.993249+0000","last_became_active":"2024-04-25T01:09:45.991011+0000","last_became_peered":"2024-04-25T01:09:45.991011+0000","last_unstale":"2024-04-25T01:09:45.993249+0000","last_undegraded":"2024-04-25T01:09:45.993249+0000","last_fullsized":"2024-04-25T01:09:45.993249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:43.390055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.955428+0000","last_change":"2024-04-25T01:09:45.943119+0000","last_active":"2024-04-25T01:09:45.955428+0000","last_peered":"2024-04-25T01:09:45.955428+0000","last_clean":"2024-04-25T01:09:45.955428+0000","last_became_active":"2024-04-25T01:09:45.942369+0000","last_became_peered":"2024-04-25T01:09:45.942369+0000","last_unstale":"2024-04-25T01:09:45.955428+0000","last_undegraded":"2024-04-25T01:09:45.955428+0000","last_fullsized":"2024-04-25T01:09:45.955428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:29.422454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989827+0000","last_change":"2024-04-25T01:09:45.974991+0000","last_active":"2024-04-25T01:09:45.989827+0000","last_peered":"2024-04-25T01:09:45.989827+0000","last_clean":"2024-04-25T01:09:45.989827+0000","last_became_active":"2024-04-25T01:09:45.974290+0000","last_became_peered":"2024-04-25T01:09:45.974290+0000","last_unstale":"2024-04-25T01:09:45.989827+0000","last_undegraded":"2024-04-25T01:09:45.989827+0000","last_fullsized":"2024-04-25T01:09:45.989827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:16.308271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.953328+0000","last_change":"2024-04-25T01:09:45.945218+0000","last_active":"2024-04-25T01:09:45.953328+0000","last_peered":"2024-04-25T01:09:45.953328+0000","last_clean":"2024-04-25T01:09:45.953328+0000","last_became_active":"2024-04-25T01:09:45.944461+0000","last_became_peered":"2024-04-25T01:09:45.944461+0000","last_unstale":"2024-04-25T01:09:45.953328+0000","last_undegraded":"2024-04-25T01:09:45.953328+0000","last_fullsized":"2024-04-25T01:09:45.953328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:31.513753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.956683+0000","last_change":"2024-04-25T01:09:45.954160+0000","last_active":"2024-04-25T01:09:45.956683+0000","last_peered":"2024-04-25T01:09:45.956683+0000","last_clean":"2024-04-25T01:09:45.956683+0000","last_became_active":"2024-04-25T01:09:45.952263+0000","last_became_peered":"2024-04-25T01:09:45.952263+0000","last_unstale":"2024-04-25T01:09:45.956683+0000","last_undegraded":"2024-04-25T01:09:45.956683+0000","last_fullsized":"2024-04-25T01:09:45.956683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:25.863316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761306+0000","last_change":"2024-04-25T01:09:45.963354+0000","last_active":"2024-04-25T01:09:47.761306+0000","last_peered":"2024-04-25T01:09:47.761306+0000","last_clean":"2024-04-25T01:09:47.761306+0000","last_became_active":"2024-04-25T01:09:45.953076+0000","last_became_peered":"2024-04-25T01:09:45.953076+0000","last_unstale":"2024-04-25T01:09:47.761306+0000","last_undegraded":"2024-04-25T01:09:47.761306+0000","last_fullsized":"2024-04-25T01:09:47.761306+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:11.144659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.947647+0000","last_change":"2024-04-25T01:09:45.938286+0000","last_active":"2024-04-25T01:09:45.947647+0000","last_peered":"2024-04-25T01:09:45.947647+0000","last_clean":"2024-04-25T01:09:45.947647+0000","last_became_active":"2024-04-25T01:09:45.937554+0000","last_became_peered":"2024-04-25T01:09:45.937554+0000","last_unstale":"2024-04-25T01:09:45.947647+0000","last_undegraded":"2024-04-25T01:09:45.947647+0000","last_fullsized":"2024-04-25T01:09:45.947647+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:35.161718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761713+0000","last_change":"2024-04-25T01:09:45.983573+0000","last_active":"2024-04-25T01:09:47.761713+0000","last_peered":"2024-04-25T01:09:47.761713+0000","last_clean":"2024-04-25T01:09:47.761713+0000","last_became_active":"2024-04-25T01:09:45.983203+0000","last_became_peered":"2024-04-25T01:09:45.983203+0000","last_unstale":"2024-04-25T01:09:47.761713+0000","last_undegraded":"2024-04-25T01:09:47.761713+0000","last_fullsized":"2024-04-25T01:09:47.761713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:17.385606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.996729+0000","last_change":"2024-04-25T01:09:44.943654+0000","last_active":"2024-04-25T01:09:45.996729+0000","last_peered":"2024-04-25T01:09:45.996729+0000","last_clean":"2024-04-25T01:09:45.996729+0000","last_became_active":"2024-04-25T01:09:44.942885+0000","last_became_peered":"2024-04-25T01:09:44.942885+0000","last_unstale":"2024-04-25T01:09:45.996729+0000","last_undegraded":"2024-04-25T01:09:45.996729+0000","last_fullsized":"2024-04-25T01:09:45.996729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:00.177175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.993154+0000","last_change":"2024-04-25T01:09:45.990382+0000","last_active":"2024-04-25T01:09:45.993154+0000","last_peered":"2024-04-25T01:09:45.993154+0000","last_clean":"2024-04-25T01:09:45.993154+0000","last_became_active":"2024-04-25T01:09:45.990064+0000","last_became_peered":"2024-04-25T01:09:45.990064+0000","last_unstale":"2024-04-25T01:09:45.993154+0000","last_undegraded":"2024-04-25T01:09:45.993154+0000","last_fullsized":"2024-04-25T01:09:45.993154+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.914637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989474+0000","last_change":"2024-04-25T01:09:44.954742+0000","last_active":"2024-04-25T01:09:45.989474+0000","last_peered":"2024-04-25T01:09:45.989474+0000","last_clean":"2024-04-25T01:09:45.989474+0000","last_became_active":"2024-04-25T01:09:44.954367+0000","last_became_peered":"2024-04-25T01:09:44.954367+0000","last_unstale":"2024-04-25T01:09:45.989474+0000","last_undegraded":"2024-04-25T01:09:45.989474+0000","last_fullsized":"2024-04-25T01:09:45.989474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:58.363142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.950061+0000","last_change":"2024-04-25T01:09:45.969566+0000","last_active":"2024-04-25T01:09:46.950061+0000","last_peered":"2024-04-25T01:09:46.950061+0000","last_clean":"2024-04-25T01:09:46.950061+0000","last_became_active":"2024-04-25T01:09:45.968874+0000","last_became_peered":"2024-04-25T01:09:45.968874+0000","last_unstale":"2024-04-25T01:09:46.950061+0000","last_undegraded":"2024-04-25T01:09:46.950061+0000","last_fullsized":"2024-04-25T01:09:46.950061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:43.384374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.978930+0000","last_change":"2024-04-25T01:09:44.929833+0000","last_active":"2024-04-25T01:09:45.978930+0000","last_peered":"2024-04-25T01:09:45.978930+0000","last_clean":"2024-04-25T01:09:45.978930+0000","last_became_active":"2024-04-25T01:09:44.929474+0000","last_became_peered":"2024-04-25T01:09:44.929474+0000","last_unstale":"2024-04-25T01:09:45.978930+0000","last_undegraded":"2024-04-25T01:09:45.978930+0000","last_fullsized":"2024-04-25T01:09:45.978930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:28.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949062+0000","last_change":"2024-04-25T01:09:45.959560+0000","last_active":"2024-04-25T01:09:46.949062+0000","last_peered":"2024-04-25T01:09:46.949062+0000","last_clean":"2024-04-25T01:09:46.949062+0000","last_became_active":"2024-04-25T01:09:45.959219+0000","last_became_peered":"2024-04-25T01:09:45.959219+0000","last_unstale":"2024-04-25T01:09:46.949062+0000","last_undegraded":"2024-04-25T01:09:46.949062+0000","last_fullsized":"2024-04-25T01:09:46.949062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:46.996714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949492+0000","last_change":"2024-04-25T01:09:44.931570+0000","last_active":"2024-04-25T01:09:46.949492+0000","last_peered":"2024-04-25T01:09:46.949492+0000","last_clean":"2024-04-25T01:09:46.949492+0000","last_became_active":"2024-04-25T01:09:44.924475+0000","last_became_peered":"2024-04-25T01:09:44.924475+0000","last_unstale":"2024-04-25T01:09:46.949492+0000","last_undegraded":"2024-04-25T01:09:46.949492+0000","last_fullsized":"2024-04-25T01:09:46.949492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:34.151431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.947855+0000","last_change":"2024-04-25T01:09:45.943598+0000","last_active":"2024-04-25T01:09:45.947855+0000","last_peered":"2024-04-25T01:09:45.947855+0000","last_clean":"2024-04-25T01:09:45.947855+0000","last_became_active":"2024-04-25T01:09:45.943194+0000","last_became_peered":"2024-04-25T01:09:45.943194+0000","last_unstale":"2024-04-25T01:09:45.947855+0000","last_undegraded":"2024-04-25T01:09:45.947855+0000","last_fullsized":"2024-04-25T01:09:45.947855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:36.501673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989551+0000","last_change":"2024-04-25T01:09:44.940411+0000","last_active":"2024-04-25T01:09:45.989551+0000","last_peered":"2024-04-25T01:09:45.989551+0000","last_clean":"2024-04-25T01:09:45.989551+0000","last_became_active":"2024-04-25T01:09:44.939598+0000","last_became_peered":"2024-04-25T01:09:44.939598+0000","last_unstale":"2024-04-25T01:09:45.989551+0000","last_undegraded":"2024-04-25T01:09:45.989551+0000","last_fullsized":"2024-04-25T01:09:45.989551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:35.491386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.982476+0000","last_change":"2024-04-25T01:09:45.959103+0000","last_active":"2024-04-25T01:09:45.982476+0000","last_peered":"2024-04-25T01:09:45.982476+0000","last_clean":"2024-04-25T01:09:45.982476+0000","last_became_active":"2024-04-25T01:09:45.958276+0000","last_became_peered":"2024-04-25T01:09:45.958276+0000","last_unstale":"2024-04-25T01:09:45.982476+0000","last_undegraded":"2024-04-25T01:09:45.982476+0000","last_fullsized":"2024-04-25T01:09:45.982476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:35.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761194+0000","last_change":"2024-04-25T01:09:44.940209+0000","last_active":"2024-04-25T01:09:47.761194+0000","last_peered":"2024-04-25T01:09:47.761194+0000","last_clean":"2024-04-25T01:09:47.761194+0000","last_became_active":"2024-04-25T01:09:44.939432+0000","last_became_peered":"2024-04-25T01:09:44.939432+0000","last_unstale":"2024-04-25T01:09:47.761194+0000","last_undegraded":"2024-04-25T01:09:47.761194+0000","last_fullsized":"2024-04-25T01:09:47.761194+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:24.853029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.950332+0000","last_change":"2024-04-25T01:09:45.982816+0000","last_active":"2024-04-25T01:09:46.950332+0000","last_peered":"2024-04-25T01:09:46.950332+0000","last_clean":"2024-04-25T01:09:46.950332+0000","last_became_active":"2024-04-25T01:09:45.976912+0000","last_became_peered":"2024-04-25T01:09:45.976912+0000","last_unstale":"2024-04-25T01:09:46.950332+0000","last_undegraded":"2024-04-25T01:09:46.950332+0000","last_fullsized":"2024-04-25T01:09:46.950332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:56.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.933690+0000","last_change":"2024-04-25T01:09:44.957432+0000","last_active":"2024-04-25T01:09:45.933690+0000","last_peered":"2024-04-25T01:09:45.933690+0000","last_clean":"2024-04-25T01:09:45.933690+0000","last_became_active":"2024-04-25T01:09:44.957144+0000","last_became_peered":"2024-04-25T01:09:44.957144+0000","last_unstale":"2024-04-25T01:09:45.933690+0000","last_undegraded":"2024-04-25T01:09:45.933690+0000","last_fullsized":"2024-04-25T01:09:45.933690+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:30.503466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948420+0000","last_change":"2024-04-25T01:09:45.990903+0000","last_active":"2024-04-25T01:09:46.948420+0000","last_peered":"2024-04-25T01:09:46.948420+0000","last_clean":"2024-04-25T01:09:46.948420+0000","last_became_active":"2024-04-25T01:09:45.990174+0000","last_became_peered":"2024-04-25T01:09:45.990174+0000","last_unstale":"2024-04-25T01:09:46.948420+0000","last_undegraded":"2024-04-25T01:09:46.948420+0000","last_fullsized":"2024-04-25T01:09:46.948420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948892+0000","last_change":"2024-04-25T01:09:44.936999+0000","last_active":"2024-04-25T01:09:46.948892+0000","last_peered":"2024-04-25T01:09:46.948892+0000","last_clean":"2024-04-25T01:09:46.948892+0000","last_became_active":"2024-04-25T01:09:44.936280+0000","last_became_peered":"2024-04-25T01:09:44.936280+0000","last_unstale":"2024-04-25T01:09:46.948892+0000","last_undegraded":"2024-04-25T01:09:46.948892+0000","last_fullsized":"2024-04-25T01:09:46.948892+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:04.012622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958162+0000","last_change":"2024-04-25T01:09:45.985444+0000","last_active":"2024-04-25T01:09:46.958162+0000","last_peered":"2024-04-25T01:09:46.958162+0000","last_clean":"2024-04-25T01:09:46.958162+0000","last_became_active":"2024-04-25T01:09:45.984992+0000","last_became_peered":"2024-04-25T01:09:45.984992+0000","last_unstale":"2024-04-25T01:09:46.958162+0000","last_undegraded":"2024-04-25T01:09:46.958162+0000","last_fullsized":"2024-04-25T01:09:46.958162+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:31.513753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949208+0000","last_change":"2024-04-25T01:09:44.928580+0000","last_active":"2024-04-25T01:09:46.949208+0000","last_peered":"2024-04-25T01:09:46.949208+0000","last_clean":"2024-04-25T01:09:46.949208+0000","last_became_active":"2024-04-25T01:09:44.928259+0000","last_became_peered":"2024-04-25T01:09:44.928259+0000","last_unstale":"2024-04-25T01:09:46.949208+0000","last_undegraded":"2024-04-25T01:09:46.949208+0000","last_fullsized":"2024-04-25T01:09:46.949208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:05.798289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761897+0000","last_change":"2024-04-25T01:09:45.974616+0000","last_active":"2024-04-25T01:09:47.761897+0000","last_peered":"2024-04-25T01:09:47.761897+0000","last_clean":"2024-04-25T01:09:47.761897+0000","last_became_active":"2024-04-25T01:09:45.964709+0000","last_became_peered":"2024-04-25T01:09:45.964709+0000","last_unstale":"2024-04-25T01:09:47.761897+0000","last_undegraded":"2024-04-25T01:09:47.761897+0000","last_fullsized":"2024-04-25T01:09:47.761897+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:52.660612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.948268+0000","last_change":"2024-04-25T01:09:44.950186+0000","last_active":"2024-04-25T01:09:45.948268+0000","last_peered":"2024-04-25T01:09:45.948268+0000","last_clean":"2024-04-25T01:09:45.948268+0000","last_became_active":"2024-04-25T01:09:44.949495+0000","last_became_peered":"2024-04-25T01:09:44.949495+0000","last_unstale":"2024-04-25T01:09:45.948268+0000","last_undegraded":"2024-04-25T01:09:45.948268+0000","last_fullsized":"2024-04-25T01:09:45.948268+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:22.800157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991878+0000","last_change":"2024-04-25T01:09:45.990498+0000","last_active":"2024-04-25T01:09:45.991878+0000","last_peered":"2024-04-25T01:09:45.991878+0000","last_clean":"2024-04-25T01:09:45.991878+0000","last_became_active":"2024-04-25T01:09:45.990172+0000","last_became_peered":"2024-04-25T01:09:45.990172+0000","last_unstale":"2024-04-25T01:09:45.991878+0000","last_undegraded":"2024-04-25T01:09:45.991878+0000","last_fullsized":"2024-04-25T01:09:45.991878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:16.308271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991041+0000","last_change":"2024-04-25T01:09:44.957706+0000","last_active":"2024-04-25T01:09:45.991041+0000","last_peered":"2024-04-25T01:09:45.991041+0000","last_clean":"2024-04-25T01:09:45.991041+0000","last_became_active":"2024-04-25T01:09:44.957237+0000","last_became_peered":"2024-04-25T01:09:44.957237+0000","last_unstale":"2024-04-25T01:09:45.991041+0000","last_undegraded":"2024-04-25T01:09:45.991041+0000","last_fullsized":"2024-04-25T01:09:45.991041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:36.582757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949726+0000","last_change":"2024-04-25T01:09:45.969865+0000","last_active":"2024-04-25T01:09:46.949726+0000","last_peered":"2024-04-25T01:09:46.949726+0000","last_clean":"2024-04-25T01:09:46.949726+0000","last_became_active":"2024-04-25T01:09:45.969105+0000","last_became_peered":"2024-04-25T01:09:45.969105+0000","last_unstale":"2024-04-25T01:09:46.949726+0000","last_undegraded":"2024-04-25T01:09:46.949726+0000","last_fullsized":"2024-04-25T01:09:46.949726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:58.181023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948623+0000","last_change":"2024-04-25T01:09:44.954726+0000","last_active":"2024-04-25T01:09:46.948623+0000","last_peered":"2024-04-25T01:09:46.948623+0000","last_clean":"2024-04-25T01:09:46.948623+0000","last_became_active":"2024-04-25T01:09:44.954400+0000","last_became_peered":"2024-04-25T01:09:44.954400+0000","last_unstale":"2024-04-25T01:09:46.948623+0000","last_undegraded":"2024-04-25T01:09:46.948623+0000","last_fullsized":"2024-04-25T01:09:46.948623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:03.154208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.981826+0000","last_change":"2024-04-25T01:09:45.981209+0000","last_active":"2024-04-25T01:09:45.981826+0000","last_peered":"2024-04-25T01:09:45.981826+0000","last_clean":"2024-04-25T01:09:45.981826+0000","last_became_active":"2024-04-25T01:09:45.980890+0000","last_became_peered":"2024-04-25T01:09:45.980890+0000","last_unstale":"2024-04-25T01:09:45.981826+0000","last_undegraded":"2024-04-25T01:09:45.981826+0000","last_fullsized":"2024-04-25T01:09:45.981826+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:37.593044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948960+0000","last_change":"2024-04-25T01:09:44.930463+0000","last_active":"2024-04-25T01:09:46.948960+0000","last_peered":"2024-04-25T01:09:46.948960+0000","last_clean":"2024-04-25T01:09:46.948960+0000","last_became_active":"2024-04-25T01:09:44.930093+0000","last_became_peered":"2024-04-25T01:09:44.930093+0000","last_unstale":"2024-04-25T01:09:46.948960+0000","last_undegraded":"2024-04-25T01:09:46.948960+0000","last_fullsized":"2024-04-25T01:09:46.948960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:42.273764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.947830+0000","last_change":"2024-04-25T01:09:45.947239+0000","last_active":"2024-04-25T01:09:46.947830+0000","last_peered":"2024-04-25T01:09:46.947830+0000","last_clean":"2024-04-25T01:09:46.947830+0000","last_became_active":"2024-04-25T01:09:45.946911+0000","last_became_peered":"2024-04-25T01:09:45.946911+0000","last_unstale":"2024-04-25T01:09:46.947830+0000","last_undegraded":"2024-04-25T01:09:46.947830+0000","last_fullsized":"2024-04-25T01:09:46.947830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:34.776822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.939452+0000","last_change":"2024-04-25T01:09:44.957202+0000","last_active":"2024-04-25T01:09:45.939452+0000","last_peered":"2024-04-25T01:09:45.939452+0000","last_clean":"2024-04-25T01:09:45.939452+0000","last_became_active":"2024-04-25T01:09:44.956782+0000","last_became_peered":"2024-04-25T01:09:44.956782+0000","last_unstale":"2024-04-25T01:09:45.939452+0000","last_undegraded":"2024-04-25T01:09:45.939452+0000","last_fullsized":"2024-04-25T01:09:45.939452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:42.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761286+0000","last_change":"2024-04-25T01:09:45.961035+0000","last_active":"2024-04-25T01:09:47.761286+0000","last_peered":"2024-04-25T01:09:47.761286+0000","last_clean":"2024-04-25T01:09:47.761286+0000","last_became_active":"2024-04-25T01:09:45.952282+0000","last_became_peered":"2024-04-25T01:09:45.952282+0000","last_unstale":"2024-04-25T01:09:47.761286+0000","last_undegraded":"2024-04-25T01:09:47.761286+0000","last_fullsized":"2024-04-25T01:09:47.761286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:26.898835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.926266+0000","last_change":"2024-04-25T01:09:44.943974+0000","last_active":"2024-04-25T01:09:45.926266+0000","last_peered":"2024-04-25T01:09:45.926266+0000","last_clean":"2024-04-25T01:09:45.926266+0000","last_became_active":"2024-04-25T01:09:44.942992+0000","last_became_peered":"2024-04-25T01:09:44.942992+0000","last_unstale":"2024-04-25T01:09:45.926266+0000","last_undegraded":"2024-04-25T01:09:45.926266+0000","last_fullsized":"2024-04-25T01:09:45.926266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:01.020214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948807+0000","last_change":"2024-04-25T01:09:45.955997+0000","last_active":"2024-04-25T01:09:46.948807+0000","last_peered":"2024-04-25T01:09:46.948807+0000","last_clean":"2024-04-25T01:09:46.948807+0000","last_became_active":"2024-04-25T01:09:45.953312+0000","last_became_peered":"2024-04-25T01:09:45.953312+0000","last_unstale":"2024-04-25T01:09:46.948807+0000","last_undegraded":"2024-04-25T01:09:46.948807+0000","last_fullsized":"2024-04-25T01:09:46.948807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:49.282873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.760941+0000","last_change":"2024-04-25T01:09:44.952579+0000","last_active":"2024-04-25T01:09:47.760941+0000","last_peered":"2024-04-25T01:09:47.760941+0000","last_clean":"2024-04-25T01:09:47.760941+0000","last_became_active":"2024-04-25T01:09:44.951862+0000","last_became_peered":"2024-04-25T01:09:44.951862+0000","last_unstale":"2024-04-25T01:09:47.760941+0000","last_undegraded":"2024-04-25T01:09:47.760941+0000","last_fullsized":"2024-04-25T01:09:47.760941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:55.605009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948680+0000","last_change":"2024-04-25T01:09:45.957482+0000","last_active":"2024-04-25T01:09:46.948680+0000","last_peered":"2024-04-25T01:09:46.948680+0000","last_clean":"2024-04-25T01:09:46.948680+0000","last_became_active":"2024-04-25T01:09:45.956793+0000","last_became_peered":"2024-04-25T01:09:45.956793+0000","last_unstale":"2024-04-25T01:09:46.948680+0000","last_undegraded":"2024-04-25T01:09:46.948680+0000","last_fullsized":"2024-04-25T01:09:46.948680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:51.065354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.996787+0000","last_change":"2024-04-25T01:09:44.948130+0000","last_active":"2024-04-25T01:09:45.996787+0000","last_peered":"2024-04-25T01:09:45.996787+0000","last_clean":"2024-04-25T01:09:45.996787+0000","last_became_active":"2024-04-25T01:09:44.947424+0000","last_became_peered":"2024-04-25T01:09:44.947424+0000","last_unstale":"2024-04-25T01:09:45.996787+0000","last_undegraded":"2024-04-25T01:09:45.996787+0000","last_fullsized":"2024-04-25T01:09:45.996787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:58.363142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992541+0000","last_change":"2024-04-25T01:09:45.988900+0000","last_active":"2024-04-25T01:09:45.992541+0000","last_peered":"2024-04-25T01:09:45.992541+0000","last_clean":"2024-04-25T01:09:45.992541+0000","last_became_active":"2024-04-25T01:09:45.987712+0000","last_became_peered":"2024-04-25T01:09:45.987712+0000","last_unstale":"2024-04-25T01:09:45.992541+0000","last_undegraded":"2024-04-25T01:09:45.992541+0000","last_fullsized":"2024-04-25T01:09:45.992541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:49.504705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949997+0000","last_change":"2024-04-25T01:09:44.951214+0000","last_active":"2024-04-25T01:09:46.949997+0000","last_peered":"2024-04-25T01:09:46.949997+0000","last_clean":"2024-04-25T01:09:46.949997+0000","last_became_active":"2024-04-25T01:09:44.950454+0000","last_became_peered":"2024-04-25T01:09:44.950454+0000","last_unstale":"2024-04-25T01:09:46.949997+0000","last_undegraded":"2024-04-25T01:09:46.949997+0000","last_fullsized":"2024-04-25T01:09:46.949997+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:30.503466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949699+0000","last_change":"2024-04-25T01:09:45.970970+0000","last_active":"2024-04-25T01:09:46.949699+0000","last_peered":"2024-04-25T01:09:46.949699+0000","last_clean":"2024-04-25T01:09:46.949699+0000","last_became_active":"2024-04-25T01:09:45.970249+0000","last_became_peered":"2024-04-25T01:09:45.970249+0000","last_unstale":"2024-04-25T01:09:46.949699+0000","last_undegraded":"2024-04-25T01:09:46.949699+0000","last_fullsized":"2024-04-25T01:09:46.949699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:04.164495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958351+0000","last_change":"2024-04-25T01:09:44.940187+0000","last_active":"2024-04-25T01:09:46.958351+0000","last_peered":"2024-04-25T01:09:46.958351+0000","last_clean":"2024-04-25T01:09:46.958351+0000","last_became_active":"2024-04-25T01:09:44.939828+0000","last_became_peered":"2024-04-25T01:09:44.939828+0000","last_unstale":"2024-04-25T01:09:46.958351+0000","last_undegraded":"2024-04-25T01:09:46.958351+0000","last_fullsized":"2024-04-25T01:09:46.958351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:44.190014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958791+0000","last_change":"2024-04-25T01:09:45.951217+0000","last_active":"2024-04-25T01:09:46.958791+0000","last_peered":"2024-04-25T01:09:46.958791+0000","last_clean":"2024-04-25T01:09:46.958791+0000","last_became_active":"2024-04-25T01:09:45.950924+0000","last_became_peered":"2024-04-25T01:09:45.950924+0000","last_unstale":"2024-04-25T01:09:46.958791+0000","last_undegraded":"2024-04-25T01:09:46.958791+0000","last_fullsized":"2024-04-25T01:09:46.958791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:25.863316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.938735+0000","last_change":"2024-04-25T01:09:44.934181+0000","last_active":"2024-04-25T01:09:45.938735+0000","last_peered":"2024-04-25T01:09:45.938735+0000","last_clean":"2024-04-25T01:09:45.938735+0000","last_became_active":"2024-04-25T01:09:44.933868+0000","last_became_peered":"2024-04-25T01:09:44.933868+0000","last_unstale":"2024-04-25T01:09:45.938735+0000","last_undegraded":"2024-04-25T01:09:45.938735+0000","last_fullsized":"2024-04-25T01:09:45.938735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:24.853029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761952+0000","last_change":"2024-04-25T01:09:45.953111+0000","last_active":"2024-04-25T01:09:47.761952+0000","last_peered":"2024-04-25T01:09:47.761952+0000","last_clean":"2024-04-25T01:09:47.761952+0000","last_became_active":"2024-04-25T01:09:45.952331+0000","last_became_peered":"2024-04-25T01:09:45.952331+0000","last_unstale":"2024-04-25T01:09:47.761952+0000","last_undegraded":"2024-04-25T01:09:47.761952+0000","last_fullsized":"2024-04-25T01:09:47.761952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:34.776822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958339+0000","last_change":"2024-04-25T01:09:44.926162+0000","last_active":"2024-04-25T01:09:46.958339+0000","last_peered":"2024-04-25T01:09:46.958339+0000","last_clean":"2024-04-25T01:09:46.958339+0000","last_became_active":"2024-04-25T01:09:44.925425+0000","last_became_peered":"2024-04-25T01:09:44.925425+0000","last_unstale":"2024-04-25T01:09:46.958339+0000","last_undegraded":"2024-04-25T01:09:46.958339+0000","last_fullsized":"2024-04-25T01:09:46.958339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:17.588543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991371+0000","last_change":"2024-04-25T01:09:45.989908+0000","last_active":"2024-04-25T01:09:45.991371+0000","last_peered":"2024-04-25T01:09:45.991371+0000","last_clean":"2024-04-25T01:09:45.991371+0000","last_became_active":"2024-04-25T01:09:45.989587+0000","last_became_peered":"2024-04-25T01:09:45.989587+0000","last_unstale":"2024-04-25T01:09:45.991371+0000","last_undegraded":"2024-04-25T01:09:45.991371+0000","last_fullsized":"2024-04-25T01:09:45.991371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:30.479872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.957923+0000","last_change":"2024-04-25T01:09:44.923407+0000","last_active":"2024-04-25T01:09:46.957923+0000","last_peered":"2024-04-25T01:09:46.957923+0000","last_clean":"2024-04-25T01:09:46.957923+0000","last_became_active":"2024-04-25T01:09:44.923041+0000","last_became_peered":"2024-04-25T01:09:44.923041+0000","last_unstale":"2024-04-25T01:09:46.957923+0000","last_undegraded":"2024-04-25T01:09:46.957923+0000","last_fullsized":"2024-04-25T01:09:46.957923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:23.364045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989715+0000","last_change":"2024-04-25T01:09:45.974584+0000","last_active":"2024-04-25T01:09:45.989715+0000","last_peered":"2024-04-25T01:09:45.989715+0000","last_clean":"2024-04-25T01:09:45.989715+0000","last_became_active":"2024-04-25T01:09:45.974075+0000","last_became_peered":"2024-04-25T01:09:45.974075+0000","last_unstale":"2024-04-25T01:09:45.989715+0000","last_undegraded":"2024-04-25T01:09:45.989715+0000","last_fullsized":"2024-04-25T01:09:45.989715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:30.479872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.762019+0000","last_change":"2024-04-25T01:09:44.949067+0000","last_active":"2024-04-25T01:09:47.762019+0000","last_peered":"2024-04-25T01:09:47.762019+0000","last_clean":"2024-04-25T01:09:47.762019+0000","last_became_active":"2024-04-25T01:09:44.948285+0000","last_became_peered":"2024-04-25T01:09:44.948285+0000","last_unstale":"2024-04-25T01:09:47.762019+0000","last_undegraded":"2024-04-25T01:09:47.762019+0000","last_fullsized":"2024-04-25T01:09:47.762019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:42.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948832+0000","last_change":"2024-04-25T01:09:45.990591+0000","last_active":"2024-04-25T01:09:46.948832+0000","last_peered":"2024-04-25T01:09:46.948832+0000","last_clean":"2024-04-25T01:09:46.948832+0000","last_became_active":"2024-04-25T01:09:45.989676+0000","last_became_peered":"2024-04-25T01:09:45.989676+0000","last_unstale":"2024-04-25T01:09:46.948832+0000","last_undegraded":"2024-04-25T01:09:46.948832+0000","last_fullsized":"2024-04-25T01:09:46.948832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:37.680178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.957558+0000","last_change":"2024-04-25T01:09:44.931330+0000","last_active":"2024-04-25T01:09:46.957558+0000","last_peered":"2024-04-25T01:09:46.957558+0000","last_clean":"2024-04-25T01:09:46.957558+0000","last_became_active":"2024-04-25T01:09:44.930909+0000","last_became_peered":"2024-04-25T01:09:44.930909+0000","last_unstale":"2024-04-25T01:09:46.957558+0000","last_undegraded":"2024-04-25T01:09:46.957558+0000","last_fullsized":"2024-04-25T01:09:46.957558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:14.765826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.980760+0000","last_change":"2024-04-25T01:09:45.976207+0000","last_active":"2024-04-25T01:09:45.980760+0000","last_peered":"2024-04-25T01:09:45.980760+0000","last_clean":"2024-04-25T01:09:45.980760+0000","last_became_active":"2024-04-25T01:09:45.975900+0000","last_became_peered":"2024-04-25T01:09:45.975900+0000","last_unstale":"2024-04-25T01:09:45.980760+0000","last_undegraded":"2024-04-25T01:09:45.980760+0000","last_fullsized":"2024-04-25T01:09:45.980760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:10.492363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992577+0000","last_change":"2024-04-25T01:09:44.957399+0000","last_active":"2024-04-25T01:09:45.992577+0000","last_peered":"2024-04-25T01:09:45.992577+0000","last_clean":"2024-04-25T01:09:45.992577+0000","last_became_active":"2024-04-25T01:09:44.956692+0000","last_became_peered":"2024-04-25T01:09:44.956692+0000","last_unstale":"2024-04-25T01:09:45.992577+0000","last_undegraded":"2024-04-25T01:09:45.992577+0000","last_fullsized":"2024-04-25T01:09:45.992577+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:13.312268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761930+0000","last_change":"2024-04-25T01:09:45.972444+0000","last_active":"2024-04-25T01:09:47.761930+0000","last_peered":"2024-04-25T01:09:47.761930+0000","last_clean":"2024-04-25T01:09:47.761930+0000","last_became_active":"2024-04-25T01:09:45.965392+0000","last_became_peered":"2024-04-25T01:09:45.965392+0000","last_unstale":"2024-04-25T01:09:47.761930+0000","last_undegraded":"2024-04-25T01:09:47.761930+0000","last_fullsized":"2024-04-25T01:09:47.761930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:47.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.961177+0000","last_change":"2024-04-25T01:09:44.939689+0000","last_active":"2024-04-25T01:09:46.961177+0000","last_peered":"2024-04-25T01:09:46.961177+0000","last_clean":"2024-04-25T01:09:46.961177+0000","last_became_active":"2024-04-25T01:09:44.939276+0000","last_became_peered":"2024-04-25T01:09:44.939276+0000","last_unstale":"2024-04-25T01:09:46.961177+0000","last_undegraded":"2024-04-25T01:09:46.961177+0000","last_fullsized":"2024-04-25T01:09:46.961177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:08.749343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948726+0000","last_change":"2024-04-25T01:09:45.979277+0000","last_active":"2024-04-25T01:09:46.948726+0000","last_peered":"2024-04-25T01:09:46.948726+0000","last_clean":"2024-04-25T01:09:46.948726+0000","last_became_active":"2024-04-25T01:09:45.978969+0000","last_became_peered":"2024-04-25T01:09:45.978969+0000","last_unstale":"2024-04-25T01:09:46.948726+0000","last_undegraded":"2024-04-25T01:09:46.948726+0000","last_fullsized":"2024-04-25T01:09:46.948726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:56.733946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.934867+0000","last_change":"2024-04-25T01:09:44.928526+0000","last_active":"2024-04-25T01:09:45.934867+0000","last_peered":"2024-04-25T01:09:45.934867+0000","last_clean":"2024-04-25T01:09:45.934867+0000","last_became_active":"2024-04-25T01:09:44.927791+0000","last_became_peered":"2024-04-25T01:09:44.927791+0000","last_unstale":"2024-04-25T01:09:45.934867+0000","last_undegraded":"2024-04-25T01:09:45.934867+0000","last_fullsized":"2024-04-25T01:09:45.934867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:56.941785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.948190+0000","last_change":"2024-04-25T01:09:45.946961+0000","last_active":"2024-04-25T01:09:45.948190+0000","last_peered":"2024-04-25T01:09:45.948190+0000","last_clean":"2024-04-25T01:09:45.948190+0000","last_became_active":"2024-04-25T01:09:45.946659+0000","last_became_peered":"2024-04-25T01:09:45.946659+0000","last_unstale":"2024-04-25T01:09:45.948190+0000","last_undegraded":"2024-04-25T01:09:45.948190+0000","last_fullsized":"2024-04-25T01:09:45.948190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:14.322556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.935279+0000","last_change":"2024-04-25T01:09:44.935837+0000","last_active":"2024-04-25T01:09:45.935279+0000","last_peered":"2024-04-25T01:09:45.935279+0000","last_clean":"2024-04-25T01:09:45.935279+0000","last_became_active":"2024-04-25T01:09:44.935376+0000","last_became_peered":"2024-04-25T01:09:44.935376+0000","last_unstale":"2024-04-25T01:09:45.935279+0000","last_undegraded":"2024-04-25T01:09:45.935279+0000","last_fullsized":"2024-04-25T01:09:45.935279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:54.518392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.948645+0000","last_change":"2024-04-25T01:09:45.945819+0000","last_active":"2024-04-25T01:09:45.948645+0000","last_peered":"2024-04-25T01:09:45.948645+0000","last_clean":"2024-04-25T01:09:45.948645+0000","last_became_active":"2024-04-25T01:09:45.945436+0000","last_became_peered":"2024-04-25T01:09:45.945436+0000","last_unstale":"2024-04-25T01:09:45.948645+0000","last_undegraded":"2024-04-25T01:09:45.948645+0000","last_fullsized":"2024-04-25T01:09:45.948645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:01.017792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.927269+0000","last_change":"2024-04-25T01:09:44.937941+0000","last_active":"2024-04-25T01:09:45.927269+0000","last_peered":"2024-04-25T01:09:45.927269+0000","last_clean":"2024-04-25T01:09:45.927269+0000","last_became_active":"2024-04-25T01:09:44.937143+0000","last_became_peered":"2024-04-25T01:09:44.937143+0000","last_unstale":"2024-04-25T01:09:45.927269+0000","last_undegraded":"2024-04-25T01:09:45.927269+0000","last_fullsized":"2024-04-25T01:09:45.927269+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:04.012622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949442+0000","last_change":"2024-04-25T01:09:45.975669+0000","last_active":"2024-04-25T01:09:46.949442+0000","last_peered":"2024-04-25T01:09:46.949442+0000","last_clean":"2024-04-25T01:09:46.949442+0000","last_became_active":"2024-04-25T01:09:45.971884+0000","last_became_peered":"2024-04-25T01:09:45.971884+0000","last_unstale":"2024-04-25T01:09:46.949442+0000","last_undegraded":"2024-04-25T01:09:46.949442+0000","last_fullsized":"2024-04-25T01:09:46.949442+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:43.284052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949100+0000","last_change":"2024-04-25T01:09:44.923165+0000","last_active":"2024-04-25T01:09:46.949100+0000","last_peered":"2024-04-25T01:09:46.949100+0000","last_clean":"2024-04-25T01:09:46.949100+0000","last_became_active":"2024-04-25T01:09:44.922477+0000","last_became_peered":"2024-04-25T01:09:44.922477+0000","last_unstale":"2024-04-25T01:09:46.949100+0000","last_undegraded":"2024-04-25T01:09:46.949100+0000","last_fullsized":"2024-04-25T01:09:46.949100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:28.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761850+0000","last_change":"2024-04-25T01:09:45.971665+0000","last_active":"2024-04-25T01:09:47.761850+0000","last_peered":"2024-04-25T01:09:47.761850+0000","last_clean":"2024-04-25T01:09:47.761850+0000","last_became_active":"2024-04-25T01:09:45.970827+0000","last_became_peered":"2024-04-25T01:09:45.970827+0000","last_unstale":"2024-04-25T01:09:47.761850+0000","last_undegraded":"2024-04-25T01:09:47.761850+0000","last_fullsized":"2024-04-25T01:09:47.761850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:49.670969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.947916+0000","last_change":"2024-04-25T01:09:44.928414+0000","last_active":"2024-04-25T01:09:46.947916+0000","last_peered":"2024-04-25T01:09:46.947916+0000","last_clean":"2024-04-25T01:09:46.947916+0000","last_became_active":"2024-04-25T01:09:44.927870+0000","last_became_peered":"2024-04-25T01:09:44.927870+0000","last_unstale":"2024-04-25T01:09:46.947916+0000","last_undegraded":"2024-04-25T01:09:46.947916+0000","last_fullsized":"2024-04-25T01:09:46.947916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:33.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992446+0000","last_change":"2024-04-25T01:09:45.976053+0000","last_active":"2024-04-25T01:09:45.992446+0000","last_peered":"2024-04-25T01:09:45.992446+0000","last_clean":"2024-04-25T01:09:45.992446+0000","last_became_active":"2024-04-25T01:09:45.960272+0000","last_became_peered":"2024-04-25T01:09:45.960272+0000","last_unstale":"2024-04-25T01:09:45.992446+0000","last_undegraded":"2024-04-25T01:09:45.992446+0000","last_fullsized":"2024-04-25T01:09:45.992446+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:15.611671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761937+0000","last_change":"2024-04-25T01:09:44.943334+0000","last_active":"2024-04-25T01:09:47.761937+0000","last_peered":"2024-04-25T01:09:47.761937+0000","last_clean":"2024-04-25T01:09:47.761937+0000","last_became_active":"2024-04-25T01:09:44.942647+0000","last_became_peered":"2024-04-25T01:09:44.942647+0000","last_unstale":"2024-04-25T01:09:47.761937+0000","last_undegraded":"2024-04-25T01:09:47.761937+0000","last_fullsized":"2024-04-25T01:09:47.761937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:00.007505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761258+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:09:47.761258+0000","last_peered":"2024-04-25T01:09:47.761258+0000","last_clean":"2024-04-25T01:09:47.761258+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:09:47.761258+0000","last_undegraded":"2024-04-25T01:09:47.761258+0000","last_fullsized":"2024-04-25T01:09:47.761258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":186125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":192839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":458752,"data_stored":193881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":185633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":178376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":768744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:09:49.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:09:50.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2e1cc7640 1 Processor -- start 2024-04-25T01:09:50.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2e1cc7640 1 -- start start 2024-04-25T01:09:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2e1cc7640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7ff2dc071a80 0x7ff2dc0b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2e1cc7640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2dc0675b0 con 0x7ff2dc15be60 2024-04-25T01:09:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2e1cc7640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2dc066ff0 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2e1cc7640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7ff2dc0b1d10 con 0x7ff2dc071a80 2024-04-25T01:09:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2db7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ff2dc15be60 0x7ff2dc0af3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53506/0 (socket says 172.21.15.33:53506) 2024-04-25T01:09:50.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2db7fe640 1 -- 172.21.15.33:0/1711974893 learned_addr learned my addr 172.21.15.33:0/1711974893 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2d2ffd640 1 --2- 172.21.15.33:0/1711974893 >> v2:172.21.15.33:3300/0 conn(0x7ff2dc071a80 0x7ff2dc0b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2daffd640 1 -- 172.21.15.33:0/1711974893 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2934478413 0 0) 0x7ff2dc0675b0 con 0x7ff2dc15be60 2024-04-25T01:09:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2daffd640 1 -- 172.21.15.33:0/1711974893 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2c8003530 con 0x7ff2dc15be60 2024-04-25T01:09:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2d2ffd640 1 -- 172.21.15.33:0/1711974893 >> v1:172.21.15.138:6789/0 conn(0x7ff2dc0b18c0 legacy=0x7ff2dc06bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2d2ffd640 1 -- 172.21.15.33:0/1711974893 >> v1:172.21.15.33:6789/0 conn(0x7ff2dc15be60 legacy=0x7ff2dc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.172+0000 7ff2d2ffd640 1 -- 172.21.15.33:0/1711974893 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc0672d0 con 0x7ff2dc071a80 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2d2ffd640 1 --2- 172.21.15.33:0/1711974893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc071a80 0x7ff2dc0b1380 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097f0 tx=0x7ff2c401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78eb1bbd43601f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2daffd640 1 -- 172.21.15.33:0/1711974893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b1070 con 0x7ff2dc071a80 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2daffd640 1 -- 172.21.15.33:0/1711974893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c40ac600 con 0x7ff2dc071a80 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2daffd640 1 -- 172.21.15.33:0/1711974893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40ac9b0 con 0x7ff2dc071a80 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/1711974893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc071a80 msgr2=0x7ff2dc0b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/1711974893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc071a80 0x7ff2dc0b1380 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40097f0 tx=0x7ff2c401c3a0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/1711974893 shutdown_connections 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/1711974893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc071a80 0x7ff2dc0b1380 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/1711974893 >> 172.21.15.33:0/1711974893 conn(0x7ff2dc0b3df0 msgr2=0x7ff2dc0b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/1711974893 shutdown_connections 2024-04-25T01:09:50.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/1711974893 wait complete. 2024-04-25T01:09:50.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 Processor -- start 2024-04-25T01:09:50.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- start start 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc071a80 0x7ff2dc14d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b18c0 0x7ff2dc0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc15be60 0x7ff2dc154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc15ca50 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc06c460 con 0x7ff2dc071a80 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2e1cc7640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc15cbd0 con 0x7ff2dc15be60 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b18c0 0x7ff2dc0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2d2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b18c0 0x7ff2dc0cd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43056/0 (socket says 172.21.15.33:43056) 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.176+0000 7ff2d2ffd640 1 -- 172.21.15.33:0/174892360 learned_addr learned my addr 172.21.15.33:0/174892360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2db7fe640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc071a80 0x7ff2dc14d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d2ffd640 1 -- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc15be60 msgr2=0x7ff2dc154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d2ffd640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc15be60 0x7ff2dc154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d2ffd640 1 -- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc071a80 msgr2=0x7ff2dc14d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d2ffd640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc071a80 0x7ff2dc14d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d2ffd640 1 -- 172.21.15.33:0/174892360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc0cc770 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d2ffd640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b18c0 0x7ff2dc0cd810 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4018fb0 tx=0x7ff2c401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d8ff9640 1 -- 172.21.15.33:0/174892360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40bc070 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d8ff9640 1 -- 172.21.15.33:0/174892360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c40b7430 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc0cca50 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d8ff9640 1 -- 172.21.15.33:0/174892360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b1040 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc0cd010 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc069040 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d8ff9640 1 -- 172.21.15.33:0/174892360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff2c40bf070 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d8ff9640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2bc0678c0 0x7ff2bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2d8ff9640 1 -- 172.21.15.33:0/174892360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff2c411f7d0 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2db7fe640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2bc0678c0 0x7ff2bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.180+0000 7ff2db7fe640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2bc0678c0 0x7ff2bc069d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc007fd0 tx=0x7ff2cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.188+0000 7ff2d8ff9640 1 -- 172.21.15.33:0/174892360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ff2c40c6080 con 0x7ff2dc0b18c0 2024-04-25T01:09:50.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.348+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff2dc0cc520 con 0x7ff2bc0678c0 2024-04-25T01:09:50.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2d8ff9640 1 -- 172.21.15.33:0/174892360 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251936 (secure 0 0 0) 0x7ff2dc0cc520 con 0x7ff2bc0678c0 2024-04-25T01:09:50.361 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:50.363 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2bc0678c0 msgr2=0x7ff2bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2bc0678c0 0x7ff2bc069d80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc007fd0 tx=0x7ff2cc009040 comp rx=0 tx=0).stop 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b18c0 msgr2=0x7ff2dc0cd810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b18c0 0x7ff2dc0cd810 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff2c4018fb0 tx=0x7ff2c401abe0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 shutdown_connections 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ff2bc0678c0 0x7ff2bc069d80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ff2dc15be60 0x7ff2dc154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2dc0b18c0 0x7ff2dc0cd810 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 --2- 172.21.15.33:0/174892360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2dc071a80 0x7ff2dc14d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.368+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 >> 172.21.15.33:0/174892360 conn(0x7ff2dc0b3df0 msgr2=0x7ff2dc06d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.372+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 shutdown_connections 2024-04-25T01:09:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.372+0000 7ff2e1cc7640 1 -- 172.21.15.33:0/174892360 wait complete. 2024-04-25T01:09:50.380 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-04-25T01:09:49.118329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229108,"kb_used_data":4596,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745612,"statfs":{"total":773094113280,"available":772859506688,"internally_reserved":0,"allocated":4706304,"data_stored":2682188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.218252"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949170+0000","last_change":"2024-04-25T01:09:45.956517+0000","last_active":"2024-04-25T01:09:46.949170+0000","last_peered":"2024-04-25T01:09:46.949170+0000","last_clean":"2024-04-25T01:09:46.949170+0000","last_became_active":"2024-04-25T01:09:45.956210+0000","last_became_peered":"2024-04-25T01:09:45.956210+0000","last_unstale":"2024-04-25T01:09:46.949170+0000","last_undegraded":"2024-04-25T01:09:46.949170+0000","last_fullsized":"2024-04-25T01:09:46.949170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:30.247260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948621+0000","last_change":"2024-04-25T01:09:45.951511+0000","last_active":"2024-04-25T01:09:46.948621+0000","last_peered":"2024-04-25T01:09:46.948621+0000","last_clean":"2024-04-25T01:09:46.948621+0000","last_became_active":"2024-04-25T01:09:45.950919+0000","last_became_peered":"2024-04-25T01:09:45.950919+0000","last_unstale":"2024-04-25T01:09:46.948621+0000","last_undegraded":"2024-04-25T01:09:46.948621+0000","last_fullsized":"2024-04-25T01:09:46.948621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:16:04.759348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991307+0000","last_change":"2024-04-25T01:09:45.989625+0000","last_active":"2024-04-25T01:09:45.991307+0000","last_peered":"2024-04-25T01:09:45.991307+0000","last_clean":"2024-04-25T01:09:45.991307+0000","last_became_active":"2024-04-25T01:09:45.989290+0000","last_became_peered":"2024-04-25T01:09:45.989290+0000","last_unstale":"2024-04-25T01:09:45.991307+0000","last_undegraded":"2024-04-25T01:09:45.991307+0000","last_fullsized":"2024-04-25T01:09:45.991307+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:17.887483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958005+0000","last_change":"2024-04-25T01:09:45.985227+0000","last_active":"2024-04-25T01:09:46.958005+0000","last_peered":"2024-04-25T01:09:46.958005+0000","last_clean":"2024-04-25T01:09:46.958005+0000","last_became_active":"2024-04-25T01:09:45.984936+0000","last_became_peered":"2024-04-25T01:09:45.984936+0000","last_unstale":"2024-04-25T01:09:46.958005+0000","last_undegraded":"2024-04-25T01:09:46.958005+0000","last_fullsized":"2024-04-25T01:09:46.958005+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:05.022909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958036+0000","last_change":"2024-04-25T01:09:45.974764+0000","last_active":"2024-04-25T01:09:46.958036+0000","last_peered":"2024-04-25T01:09:46.958036+0000","last_clean":"2024-04-25T01:09:46.958036+0000","last_became_active":"2024-04-25T01:09:45.974517+0000","last_became_peered":"2024-04-25T01:09:45.974517+0000","last_unstale":"2024-04-25T01:09:46.958036+0000","last_undegraded":"2024-04-25T01:09:46.958036+0000","last_fullsized":"2024-04-25T01:09:46.958036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:29.422454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.955016+0000","last_change":"2024-04-25T01:09:45.952003+0000","last_active":"2024-04-25T01:09:45.955016+0000","last_peered":"2024-04-25T01:09:45.955016+0000","last_clean":"2024-04-25T01:09:45.955016+0000","last_became_active":"2024-04-25T01:09:45.951686+0000","last_became_peered":"2024-04-25T01:09:45.951686+0000","last_unstale":"2024-04-25T01:09:45.955016+0000","last_undegraded":"2024-04-25T01:09:45.955016+0000","last_fullsized":"2024-04-25T01:09:45.955016+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:56.615296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989992+0000","last_change":"2024-04-25T01:09:45.976515+0000","last_active":"2024-04-25T01:09:45.989992+0000","last_peered":"2024-04-25T01:09:45.989992+0000","last_clean":"2024-04-25T01:09:45.989992+0000","last_became_active":"2024-04-25T01:09:45.960237+0000","last_became_peered":"2024-04-25T01:09:45.960237+0000","last_unstale":"2024-04-25T01:09:45.989992+0000","last_undegraded":"2024-04-25T01:09:45.989992+0000","last_fullsized":"2024-04-25T01:09:45.989992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:27:38.105948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.947796+0000","last_change":"2024-04-25T01:09:45.984500+0000","last_active":"2024-04-25T01:09:46.947796+0000","last_peered":"2024-04-25T01:09:46.947796+0000","last_clean":"2024-04-25T01:09:46.947796+0000","last_became_active":"2024-04-25T01:09:45.984132+0000","last_became_peered":"2024-04-25T01:09:45.984132+0000","last_unstale":"2024-04-25T01:09:46.947796+0000","last_undegraded":"2024-04-25T01:09:46.947796+0000","last_fullsized":"2024-04-25T01:09:46.947796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:47.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.987523+0000","last_change":"2024-04-25T01:09:45.986604+0000","last_active":"2024-04-25T01:09:45.987523+0000","last_peered":"2024-04-25T01:09:45.987523+0000","last_clean":"2024-04-25T01:09:45.987523+0000","last_became_active":"2024-04-25T01:09:45.986334+0000","last_became_peered":"2024-04-25T01:09:45.986334+0000","last_unstale":"2024-04-25T01:09:45.987523+0000","last_undegraded":"2024-04-25T01:09:45.987523+0000","last_fullsized":"2024-04-25T01:09:45.987523+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:35.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992385+0000","last_change":"2024-04-25T01:09:45.987850+0000","last_active":"2024-04-25T01:09:45.992385+0000","last_peered":"2024-04-25T01:09:45.992385+0000","last_clean":"2024-04-25T01:09:45.992385+0000","last_became_active":"2024-04-25T01:09:45.987281+0000","last_became_peered":"2024-04-25T01:09:45.987281+0000","last_unstale":"2024-04-25T01:09:45.992385+0000","last_undegraded":"2024-04-25T01:09:45.992385+0000","last_fullsized":"2024-04-25T01:09:45.992385+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:14.111400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992346+0000","last_change":"2024-04-25T01:09:45.987178+0000","last_active":"2024-04-25T01:09:45.992346+0000","last_peered":"2024-04-25T01:09:45.992346+0000","last_clean":"2024-04-25T01:09:45.992346+0000","last_became_active":"2024-04-25T01:09:45.983271+0000","last_became_peered":"2024-04-25T01:09:45.983271+0000","last_unstale":"2024-04-25T01:09:45.992346+0000","last_undegraded":"2024-04-25T01:09:45.992346+0000","last_fullsized":"2024-04-25T01:09:45.992346+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:07.230148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.953791+0000","last_change":"2024-04-25T01:09:45.949145+0000","last_active":"2024-04-25T01:09:45.953791+0000","last_peered":"2024-04-25T01:09:45.953791+0000","last_clean":"2024-04-25T01:09:45.953791+0000","last_became_active":"2024-04-25T01:09:45.948672+0000","last_became_peered":"2024-04-25T01:09:45.948672+0000","last_unstale":"2024-04-25T01:09:45.953791+0000","last_undegraded":"2024-04-25T01:09:45.953791+0000","last_fullsized":"2024-04-25T01:09:45.953791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:23.810444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989856+0000","last_change":"2024-04-25T01:09:45.977243+0000","last_active":"2024-04-25T01:09:45.989856+0000","last_peered":"2024-04-25T01:09:45.989856+0000","last_clean":"2024-04-25T01:09:45.989856+0000","last_became_active":"2024-04-25T01:09:45.976641+0000","last_became_peered":"2024-04-25T01:09:45.976641+0000","last_unstale":"2024-04-25T01:09:45.989856+0000","last_undegraded":"2024-04-25T01:09:45.989856+0000","last_fullsized":"2024-04-25T01:09:45.989856+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:51.578818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949053+0000","last_change":"2024-04-25T01:09:45.973163+0000","last_active":"2024-04-25T01:09:46.949053+0000","last_peered":"2024-04-25T01:09:46.949053+0000","last_clean":"2024-04-25T01:09:46.949053+0000","last_became_active":"2024-04-25T01:09:45.971174+0000","last_became_peered":"2024-04-25T01:09:45.971174+0000","last_unstale":"2024-04-25T01:09:46.949053+0000","last_undegraded":"2024-04-25T01:09:46.949053+0000","last_fullsized":"2024-04-25T01:09:46.949053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:34.705500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.982532+0000","last_change":"2024-04-25T01:09:45.959115+0000","last_active":"2024-04-25T01:09:45.982532+0000","last_peered":"2024-04-25T01:09:45.982532+0000","last_clean":"2024-04-25T01:09:45.982532+0000","last_became_active":"2024-04-25T01:09:45.958380+0000","last_became_peered":"2024-04-25T01:09:45.958380+0000","last_unstale":"2024-04-25T01:09:45.982532+0000","last_undegraded":"2024-04-25T01:09:45.982532+0000","last_fullsized":"2024-04-25T01:09:45.982532+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:13:17.887483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.760930+0000","last_change":"2024-04-25T01:09:45.992991+0000","last_active":"2024-04-25T01:09:47.760930+0000","last_peered":"2024-04-25T01:09:47.760930+0000","last_clean":"2024-04-25T01:09:47.760930+0000","last_became_active":"2024-04-25T01:09:45.990811+0000","last_became_peered":"2024-04-25T01:09:45.990811+0000","last_unstale":"2024-04-25T01:09:47.760930+0000","last_undegraded":"2024-04-25T01:09:47.760930+0000","last_fullsized":"2024-04-25T01:09:47.760930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:46.283959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.762050+0000","last_change":"2024-04-25T01:09:45.987765+0000","last_active":"2024-04-25T01:09:47.762050+0000","last_peered":"2024-04-25T01:09:47.762050+0000","last_clean":"2024-04-25T01:09:47.762050+0000","last_became_active":"2024-04-25T01:09:45.987230+0000","last_became_peered":"2024-04-25T01:09:45.987230+0000","last_unstale":"2024-04-25T01:09:47.762050+0000","last_undegraded":"2024-04-25T01:09:47.762050+0000","last_fullsized":"2024-04-25T01:09:47.762050+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.976089+0000","last_change":"2024-04-25T01:09:45.974198+0000","last_active":"2024-04-25T01:09:45.976089+0000","last_peered":"2024-04-25T01:09:45.976089+0000","last_clean":"2024-04-25T01:09:45.976089+0000","last_became_active":"2024-04-25T01:09:45.973480+0000","last_became_peered":"2024-04-25T01:09:45.973480+0000","last_unstale":"2024-04-25T01:09:45.976089+0000","last_undegraded":"2024-04-25T01:09:45.976089+0000","last_fullsized":"2024-04-25T01:09:45.976089+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:43.736020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958359+0000","last_change":"2024-04-25T01:09:45.943626+0000","last_active":"2024-04-25T01:09:46.958359+0000","last_peered":"2024-04-25T01:09:46.958359+0000","last_clean":"2024-04-25T01:09:46.958359+0000","last_became_active":"2024-04-25T01:09:45.943334+0000","last_became_peered":"2024-04-25T01:09:45.943334+0000","last_unstale":"2024-04-25T01:09:46.958359+0000","last_undegraded":"2024-04-25T01:09:46.958359+0000","last_fullsized":"2024-04-25T01:09:46.958359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:35.161718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.985995+0000","last_change":"2024-04-25T01:09:45.981951+0000","last_active":"2024-04-25T01:09:45.985995+0000","last_peered":"2024-04-25T01:09:45.985995+0000","last_clean":"2024-04-25T01:09:45.985995+0000","last_became_active":"2024-04-25T01:09:45.981250+0000","last_became_peered":"2024-04-25T01:09:45.981250+0000","last_unstale":"2024-04-25T01:09:45.985995+0000","last_undegraded":"2024-04-25T01:09:45.985995+0000","last_fullsized":"2024-04-25T01:09:45.985995+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:01.017792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989925+0000","last_change":"2024-04-25T01:09:45.980355+0000","last_active":"2024-04-25T01:09:45.989925+0000","last_peered":"2024-04-25T01:09:45.989925+0000","last_clean":"2024-04-25T01:09:45.989925+0000","last_became_active":"2024-04-25T01:09:45.950989+0000","last_became_peered":"2024-04-25T01:09:45.950989+0000","last_unstale":"2024-04-25T01:09:45.989925+0000","last_undegraded":"2024-04-25T01:09:45.989925+0000","last_fullsized":"2024-04-25T01:09:45.989925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:52.298327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949972+0000","last_change":"2024-04-25T01:09:45.972529+0000","last_active":"2024-04-25T01:09:46.949972+0000","last_peered":"2024-04-25T01:09:46.949972+0000","last_clean":"2024-04-25T01:09:46.949972+0000","last_became_active":"2024-04-25T01:09:45.970396+0000","last_became_peered":"2024-04-25T01:09:45.970396+0000","last_unstale":"2024-04-25T01:09:46.949972+0000","last_undegraded":"2024-04-25T01:09:46.949972+0000","last_fullsized":"2024-04-25T01:09:46.949972+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:06.808576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761308+0000","last_change":"2024-04-25T01:09:45.962771+0000","last_active":"2024-04-25T01:09:47.761308+0000","last_peered":"2024-04-25T01:09:47.761308+0000","last_clean":"2024-04-25T01:09:47.761308+0000","last_became_active":"2024-04-25T01:09:45.952747+0000","last_became_peered":"2024-04-25T01:09:45.952747+0000","last_unstale":"2024-04-25T01:09:47.761308+0000","last_undegraded":"2024-04-25T01:09:47.761308+0000","last_fullsized":"2024-04-25T01:09:47.761308+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:57:17.084255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.957754+0000","last_change":"2024-04-25T01:09:45.956306+0000","last_active":"2024-04-25T01:09:46.957754+0000","last_peered":"2024-04-25T01:09:46.957754+0000","last_clean":"2024-04-25T01:09:46.957754+0000","last_became_active":"2024-04-25T01:09:45.954871+0000","last_became_peered":"2024-04-25T01:09:45.954871+0000","last_unstale":"2024-04-25T01:09:46.957754+0000","last_undegraded":"2024-04-25T01:09:46.957754+0000","last_fullsized":"2024-04-25T01:09:46.957754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:01.187462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.957376+0000","last_change":"2024-04-25T01:09:45.945731+0000","last_active":"2024-04-25T01:09:45.957376+0000","last_peered":"2024-04-25T01:09:45.957376+0000","last_clean":"2024-04-25T01:09:45.957376+0000","last_became_active":"2024-04-25T01:09:45.945040+0000","last_became_peered":"2024-04-25T01:09:45.945040+0000","last_unstale":"2024-04-25T01:09:45.957376+0000","last_undegraded":"2024-04-25T01:09:45.957376+0000","last_fullsized":"2024-04-25T01:09:45.957376+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:35.161718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.993249+0000","last_change":"2024-04-25T01:09:45.991405+0000","last_active":"2024-04-25T01:09:45.993249+0000","last_peered":"2024-04-25T01:09:45.993249+0000","last_clean":"2024-04-25T01:09:45.993249+0000","last_became_active":"2024-04-25T01:09:45.991011+0000","last_became_peered":"2024-04-25T01:09:45.991011+0000","last_unstale":"2024-04-25T01:09:45.993249+0000","last_undegraded":"2024-04-25T01:09:45.993249+0000","last_fullsized":"2024-04-25T01:09:45.993249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:15:43.390055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.955428+0000","last_change":"2024-04-25T01:09:45.943119+0000","last_active":"2024-04-25T01:09:45.955428+0000","last_peered":"2024-04-25T01:09:45.955428+0000","last_clean":"2024-04-25T01:09:45.955428+0000","last_became_active":"2024-04-25T01:09:45.942369+0000","last_became_peered":"2024-04-25T01:09:45.942369+0000","last_unstale":"2024-04-25T01:09:45.955428+0000","last_undegraded":"2024-04-25T01:09:45.955428+0000","last_fullsized":"2024-04-25T01:09:45.955428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:29.422454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989827+0000","last_change":"2024-04-25T01:09:45.974991+0000","last_active":"2024-04-25T01:09:45.989827+0000","last_peered":"2024-04-25T01:09:45.989827+0000","last_clean":"2024-04-25T01:09:45.989827+0000","last_became_active":"2024-04-25T01:09:45.974290+0000","last_became_peered":"2024-04-25T01:09:45.974290+0000","last_unstale":"2024-04-25T01:09:45.989827+0000","last_undegraded":"2024-04-25T01:09:45.989827+0000","last_fullsized":"2024-04-25T01:09:45.989827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:16.308271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.953328+0000","last_change":"2024-04-25T01:09:45.945218+0000","last_active":"2024-04-25T01:09:45.953328+0000","last_peered":"2024-04-25T01:09:45.953328+0000","last_clean":"2024-04-25T01:09:45.953328+0000","last_became_active":"2024-04-25T01:09:45.944461+0000","last_became_peered":"2024-04-25T01:09:45.944461+0000","last_unstale":"2024-04-25T01:09:45.953328+0000","last_undegraded":"2024-04-25T01:09:45.953328+0000","last_fullsized":"2024-04-25T01:09:45.953328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:31.513753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.956683+0000","last_change":"2024-04-25T01:09:45.954160+0000","last_active":"2024-04-25T01:09:45.956683+0000","last_peered":"2024-04-25T01:09:45.956683+0000","last_clean":"2024-04-25T01:09:45.956683+0000","last_became_active":"2024-04-25T01:09:45.952263+0000","last_became_peered":"2024-04-25T01:09:45.952263+0000","last_unstale":"2024-04-25T01:09:45.956683+0000","last_undegraded":"2024-04-25T01:09:45.956683+0000","last_fullsized":"2024-04-25T01:09:45.956683+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:25.863316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761306+0000","last_change":"2024-04-25T01:09:45.963354+0000","last_active":"2024-04-25T01:09:47.761306+0000","last_peered":"2024-04-25T01:09:47.761306+0000","last_clean":"2024-04-25T01:09:47.761306+0000","last_became_active":"2024-04-25T01:09:45.953076+0000","last_became_peered":"2024-04-25T01:09:45.953076+0000","last_unstale":"2024-04-25T01:09:47.761306+0000","last_undegraded":"2024-04-25T01:09:47.761306+0000","last_fullsized":"2024-04-25T01:09:47.761306+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:54:11.144659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.947647+0000","last_change":"2024-04-25T01:09:45.938286+0000","last_active":"2024-04-25T01:09:45.947647+0000","last_peered":"2024-04-25T01:09:45.947647+0000","last_clean":"2024-04-25T01:09:45.947647+0000","last_became_active":"2024-04-25T01:09:45.937554+0000","last_became_peered":"2024-04-25T01:09:45.937554+0000","last_unstale":"2024-04-25T01:09:45.947647+0000","last_undegraded":"2024-04-25T01:09:45.947647+0000","last_fullsized":"2024-04-25T01:09:45.947647+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:35.161718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761713+0000","last_change":"2024-04-25T01:09:45.983573+0000","last_active":"2024-04-25T01:09:47.761713+0000","last_peered":"2024-04-25T01:09:47.761713+0000","last_clean":"2024-04-25T01:09:47.761713+0000","last_became_active":"2024-04-25T01:09:45.983203+0000","last_became_peered":"2024-04-25T01:09:45.983203+0000","last_unstale":"2024-04-25T01:09:47.761713+0000","last_undegraded":"2024-04-25T01:09:47.761713+0000","last_fullsized":"2024-04-25T01:09:47.761713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:17.385606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.996729+0000","last_change":"2024-04-25T01:09:44.943654+0000","last_active":"2024-04-25T01:09:45.996729+0000","last_peered":"2024-04-25T01:09:45.996729+0000","last_clean":"2024-04-25T01:09:45.996729+0000","last_became_active":"2024-04-25T01:09:44.942885+0000","last_became_peered":"2024-04-25T01:09:44.942885+0000","last_unstale":"2024-04-25T01:09:45.996729+0000","last_undegraded":"2024-04-25T01:09:45.996729+0000","last_fullsized":"2024-04-25T01:09:45.996729+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:00.177175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.993154+0000","last_change":"2024-04-25T01:09:45.990382+0000","last_active":"2024-04-25T01:09:45.993154+0000","last_peered":"2024-04-25T01:09:45.993154+0000","last_clean":"2024-04-25T01:09:45.993154+0000","last_became_active":"2024-04-25T01:09:45.990064+0000","last_became_peered":"2024-04-25T01:09:45.990064+0000","last_unstale":"2024-04-25T01:09:45.993154+0000","last_undegraded":"2024-04-25T01:09:45.993154+0000","last_fullsized":"2024-04-25T01:09:45.993154+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:18:10.914637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989474+0000","last_change":"2024-04-25T01:09:44.954742+0000","last_active":"2024-04-25T01:09:45.989474+0000","last_peered":"2024-04-25T01:09:45.989474+0000","last_clean":"2024-04-25T01:09:45.989474+0000","last_became_active":"2024-04-25T01:09:44.954367+0000","last_became_peered":"2024-04-25T01:09:44.954367+0000","last_unstale":"2024-04-25T01:09:45.989474+0000","last_undegraded":"2024-04-25T01:09:45.989474+0000","last_fullsized":"2024-04-25T01:09:45.989474+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:58.363142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.950061+0000","last_change":"2024-04-25T01:09:45.969566+0000","last_active":"2024-04-25T01:09:46.950061+0000","last_peered":"2024-04-25T01:09:46.950061+0000","last_clean":"2024-04-25T01:09:46.950061+0000","last_became_active":"2024-04-25T01:09:45.968874+0000","last_became_peered":"2024-04-25T01:09:45.968874+0000","last_unstale":"2024-04-25T01:09:46.950061+0000","last_undegraded":"2024-04-25T01:09:46.950061+0000","last_fullsized":"2024-04-25T01:09:46.950061+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:43.384374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.978930+0000","last_change":"2024-04-25T01:09:44.929833+0000","last_active":"2024-04-25T01:09:45.978930+0000","last_peered":"2024-04-25T01:09:45.978930+0000","last_clean":"2024-04-25T01:09:45.978930+0000","last_became_active":"2024-04-25T01:09:44.929474+0000","last_became_peered":"2024-04-25T01:09:44.929474+0000","last_unstale":"2024-04-25T01:09:45.978930+0000","last_undegraded":"2024-04-25T01:09:45.978930+0000","last_fullsized":"2024-04-25T01:09:45.978930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:28.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949062+0000","last_change":"2024-04-25T01:09:45.959560+0000","last_active":"2024-04-25T01:09:46.949062+0000","last_peered":"2024-04-25T01:09:46.949062+0000","last_clean":"2024-04-25T01:09:46.949062+0000","last_became_active":"2024-04-25T01:09:45.959219+0000","last_became_peered":"2024-04-25T01:09:45.959219+0000","last_unstale":"2024-04-25T01:09:46.949062+0000","last_undegraded":"2024-04-25T01:09:46.949062+0000","last_fullsized":"2024-04-25T01:09:46.949062+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:12:46.996714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949492+0000","last_change":"2024-04-25T01:09:44.931570+0000","last_active":"2024-04-25T01:09:46.949492+0000","last_peered":"2024-04-25T01:09:46.949492+0000","last_clean":"2024-04-25T01:09:46.949492+0000","last_became_active":"2024-04-25T01:09:44.924475+0000","last_became_peered":"2024-04-25T01:09:44.924475+0000","last_unstale":"2024-04-25T01:09:46.949492+0000","last_undegraded":"2024-04-25T01:09:46.949492+0000","last_fullsized":"2024-04-25T01:09:46.949492+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:51:34.151431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.947855+0000","last_change":"2024-04-25T01:09:45.943598+0000","last_active":"2024-04-25T01:09:45.947855+0000","last_peered":"2024-04-25T01:09:45.947855+0000","last_clean":"2024-04-25T01:09:45.947855+0000","last_became_active":"2024-04-25T01:09:45.943194+0000","last_became_peered":"2024-04-25T01:09:45.943194+0000","last_unstale":"2024-04-25T01:09:45.947855+0000","last_undegraded":"2024-04-25T01:09:45.947855+0000","last_fullsized":"2024-04-25T01:09:45.947855+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:36.501673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989551+0000","last_change":"2024-04-25T01:09:44.940411+0000","last_active":"2024-04-25T01:09:45.989551+0000","last_peered":"2024-04-25T01:09:45.989551+0000","last_clean":"2024-04-25T01:09:45.989551+0000","last_became_active":"2024-04-25T01:09:44.939598+0000","last_became_peered":"2024-04-25T01:09:44.939598+0000","last_unstale":"2024-04-25T01:09:45.989551+0000","last_undegraded":"2024-04-25T01:09:45.989551+0000","last_fullsized":"2024-04-25T01:09:45.989551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:35.491386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.982476+0000","last_change":"2024-04-25T01:09:45.959103+0000","last_active":"2024-04-25T01:09:45.982476+0000","last_peered":"2024-04-25T01:09:45.982476+0000","last_clean":"2024-04-25T01:09:45.982476+0000","last_became_active":"2024-04-25T01:09:45.958276+0000","last_became_peered":"2024-04-25T01:09:45.958276+0000","last_unstale":"2024-04-25T01:09:45.982476+0000","last_undegraded":"2024-04-25T01:09:45.982476+0000","last_fullsized":"2024-04-25T01:09:45.982476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:15:35.913764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761194+0000","last_change":"2024-04-25T01:09:44.940209+0000","last_active":"2024-04-25T01:09:47.761194+0000","last_peered":"2024-04-25T01:09:47.761194+0000","last_clean":"2024-04-25T01:09:47.761194+0000","last_became_active":"2024-04-25T01:09:44.939432+0000","last_became_peered":"2024-04-25T01:09:44.939432+0000","last_unstale":"2024-04-25T01:09:47.761194+0000","last_undegraded":"2024-04-25T01:09:47.761194+0000","last_fullsized":"2024-04-25T01:09:47.761194+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:24.853029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.950332+0000","last_change":"2024-04-25T01:09:45.982816+0000","last_active":"2024-04-25T01:09:46.950332+0000","last_peered":"2024-04-25T01:09:46.950332+0000","last_clean":"2024-04-25T01:09:46.950332+0000","last_became_active":"2024-04-25T01:09:45.976912+0000","last_became_peered":"2024-04-25T01:09:45.976912+0000","last_unstale":"2024-04-25T01:09:46.950332+0000","last_undegraded":"2024-04-25T01:09:46.950332+0000","last_fullsized":"2024-04-25T01:09:46.950332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:56.021671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.933690+0000","last_change":"2024-04-25T01:09:44.957432+0000","last_active":"2024-04-25T01:09:45.933690+0000","last_peered":"2024-04-25T01:09:45.933690+0000","last_clean":"2024-04-25T01:09:45.933690+0000","last_became_active":"2024-04-25T01:09:44.957144+0000","last_became_peered":"2024-04-25T01:09:44.957144+0000","last_unstale":"2024-04-25T01:09:45.933690+0000","last_undegraded":"2024-04-25T01:09:45.933690+0000","last_fullsized":"2024-04-25T01:09:45.933690+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:30.503466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948420+0000","last_change":"2024-04-25T01:09:45.990903+0000","last_active":"2024-04-25T01:09:46.948420+0000","last_peered":"2024-04-25T01:09:46.948420+0000","last_clean":"2024-04-25T01:09:46.948420+0000","last_became_active":"2024-04-25T01:09:45.990174+0000","last_became_peered":"2024-04-25T01:09:45.990174+0000","last_unstale":"2024-04-25T01:09:46.948420+0000","last_undegraded":"2024-04-25T01:09:46.948420+0000","last_fullsized":"2024-04-25T01:09:46.948420+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:36.609946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948892+0000","last_change":"2024-04-25T01:09:44.936999+0000","last_active":"2024-04-25T01:09:46.948892+0000","last_peered":"2024-04-25T01:09:46.948892+0000","last_clean":"2024-04-25T01:09:46.948892+0000","last_became_active":"2024-04-25T01:09:44.936280+0000","last_became_peered":"2024-04-25T01:09:44.936280+0000","last_unstale":"2024-04-25T01:09:46.948892+0000","last_undegraded":"2024-04-25T01:09:46.948892+0000","last_fullsized":"2024-04-25T01:09:46.948892+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:04.012622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958162+0000","last_change":"2024-04-25T01:09:45.985444+0000","last_active":"2024-04-25T01:09:46.958162+0000","last_peered":"2024-04-25T01:09:46.958162+0000","last_clean":"2024-04-25T01:09:46.958162+0000","last_became_active":"2024-04-25T01:09:45.984992+0000","last_became_peered":"2024-04-25T01:09:45.984992+0000","last_unstale":"2024-04-25T01:09:46.958162+0000","last_undegraded":"2024-04-25T01:09:46.958162+0000","last_fullsized":"2024-04-25T01:09:46.958162+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:31.513753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949208+0000","last_change":"2024-04-25T01:09:44.928580+0000","last_active":"2024-04-25T01:09:46.949208+0000","last_peered":"2024-04-25T01:09:46.949208+0000","last_clean":"2024-04-25T01:09:46.949208+0000","last_became_active":"2024-04-25T01:09:44.928259+0000","last_became_peered":"2024-04-25T01:09:44.928259+0000","last_unstale":"2024-04-25T01:09:46.949208+0000","last_undegraded":"2024-04-25T01:09:46.949208+0000","last_fullsized":"2024-04-25T01:09:46.949208+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:25:05.798289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761897+0000","last_change":"2024-04-25T01:09:45.974616+0000","last_active":"2024-04-25T01:09:47.761897+0000","last_peered":"2024-04-25T01:09:47.761897+0000","last_clean":"2024-04-25T01:09:47.761897+0000","last_became_active":"2024-04-25T01:09:45.964709+0000","last_became_peered":"2024-04-25T01:09:45.964709+0000","last_unstale":"2024-04-25T01:09:47.761897+0000","last_undegraded":"2024-04-25T01:09:47.761897+0000","last_fullsized":"2024-04-25T01:09:47.761897+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:44:52.660612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.948268+0000","last_change":"2024-04-25T01:09:44.950186+0000","last_active":"2024-04-25T01:09:45.948268+0000","last_peered":"2024-04-25T01:09:45.948268+0000","last_clean":"2024-04-25T01:09:45.948268+0000","last_became_active":"2024-04-25T01:09:44.949495+0000","last_became_peered":"2024-04-25T01:09:44.949495+0000","last_unstale":"2024-04-25T01:09:45.948268+0000","last_undegraded":"2024-04-25T01:09:45.948268+0000","last_fullsized":"2024-04-25T01:09:45.948268+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:11:22.800157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991878+0000","last_change":"2024-04-25T01:09:45.990498+0000","last_active":"2024-04-25T01:09:45.991878+0000","last_peered":"2024-04-25T01:09:45.991878+0000","last_clean":"2024-04-25T01:09:45.991878+0000","last_became_active":"2024-04-25T01:09:45.990172+0000","last_became_peered":"2024-04-25T01:09:45.990172+0000","last_unstale":"2024-04-25T01:09:45.991878+0000","last_undegraded":"2024-04-25T01:09:45.991878+0000","last_fullsized":"2024-04-25T01:09:45.991878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:09:16.308271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991041+0000","last_change":"2024-04-25T01:09:44.957706+0000","last_active":"2024-04-25T01:09:45.991041+0000","last_peered":"2024-04-25T01:09:45.991041+0000","last_clean":"2024-04-25T01:09:45.991041+0000","last_became_active":"2024-04-25T01:09:44.957237+0000","last_became_peered":"2024-04-25T01:09:44.957237+0000","last_unstale":"2024-04-25T01:09:45.991041+0000","last_undegraded":"2024-04-25T01:09:45.991041+0000","last_fullsized":"2024-04-25T01:09:45.991041+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:36.582757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949726+0000","last_change":"2024-04-25T01:09:45.969865+0000","last_active":"2024-04-25T01:09:46.949726+0000","last_peered":"2024-04-25T01:09:46.949726+0000","last_clean":"2024-04-25T01:09:46.949726+0000","last_became_active":"2024-04-25T01:09:45.969105+0000","last_became_peered":"2024-04-25T01:09:45.969105+0000","last_unstale":"2024-04-25T01:09:46.949726+0000","last_undegraded":"2024-04-25T01:09:46.949726+0000","last_fullsized":"2024-04-25T01:09:46.949726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:32:58.181023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948623+0000","last_change":"2024-04-25T01:09:44.954726+0000","last_active":"2024-04-25T01:09:46.948623+0000","last_peered":"2024-04-25T01:09:46.948623+0000","last_clean":"2024-04-25T01:09:46.948623+0000","last_became_active":"2024-04-25T01:09:44.954400+0000","last_became_peered":"2024-04-25T01:09:44.954400+0000","last_unstale":"2024-04-25T01:09:46.948623+0000","last_undegraded":"2024-04-25T01:09:46.948623+0000","last_fullsized":"2024-04-25T01:09:46.948623+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:03.154208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.981826+0000","last_change":"2024-04-25T01:09:45.981209+0000","last_active":"2024-04-25T01:09:45.981826+0000","last_peered":"2024-04-25T01:09:45.981826+0000","last_clean":"2024-04-25T01:09:45.981826+0000","last_became_active":"2024-04-25T01:09:45.980890+0000","last_became_peered":"2024-04-25T01:09:45.980890+0000","last_unstale":"2024-04-25T01:09:45.981826+0000","last_undegraded":"2024-04-25T01:09:45.981826+0000","last_fullsized":"2024-04-25T01:09:45.981826+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:37.593044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948960+0000","last_change":"2024-04-25T01:09:44.930463+0000","last_active":"2024-04-25T01:09:46.948960+0000","last_peered":"2024-04-25T01:09:46.948960+0000","last_clean":"2024-04-25T01:09:46.948960+0000","last_became_active":"2024-04-25T01:09:44.930093+0000","last_became_peered":"2024-04-25T01:09:44.930093+0000","last_unstale":"2024-04-25T01:09:46.948960+0000","last_undegraded":"2024-04-25T01:09:46.948960+0000","last_fullsized":"2024-04-25T01:09:46.948960+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:42.273764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.947830+0000","last_change":"2024-04-25T01:09:45.947239+0000","last_active":"2024-04-25T01:09:46.947830+0000","last_peered":"2024-04-25T01:09:46.947830+0000","last_clean":"2024-04-25T01:09:46.947830+0000","last_became_active":"2024-04-25T01:09:45.946911+0000","last_became_peered":"2024-04-25T01:09:45.946911+0000","last_unstale":"2024-04-25T01:09:46.947830+0000","last_undegraded":"2024-04-25T01:09:46.947830+0000","last_fullsized":"2024-04-25T01:09:46.947830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:34.776822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.939452+0000","last_change":"2024-04-25T01:09:44.957202+0000","last_active":"2024-04-25T01:09:45.939452+0000","last_peered":"2024-04-25T01:09:45.939452+0000","last_clean":"2024-04-25T01:09:45.939452+0000","last_became_active":"2024-04-25T01:09:44.956782+0000","last_became_peered":"2024-04-25T01:09:44.956782+0000","last_unstale":"2024-04-25T01:09:45.939452+0000","last_undegraded":"2024-04-25T01:09:45.939452+0000","last_fullsized":"2024-04-25T01:09:45.939452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:42.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761286+0000","last_change":"2024-04-25T01:09:45.961035+0000","last_active":"2024-04-25T01:09:47.761286+0000","last_peered":"2024-04-25T01:09:47.761286+0000","last_clean":"2024-04-25T01:09:47.761286+0000","last_became_active":"2024-04-25T01:09:45.952282+0000","last_became_peered":"2024-04-25T01:09:45.952282+0000","last_unstale":"2024-04-25T01:09:47.761286+0000","last_undegraded":"2024-04-25T01:09:47.761286+0000","last_fullsized":"2024-04-25T01:09:47.761286+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:02:26.898835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.926266+0000","last_change":"2024-04-25T01:09:44.943974+0000","last_active":"2024-04-25T01:09:45.926266+0000","last_peered":"2024-04-25T01:09:45.926266+0000","last_clean":"2024-04-25T01:09:45.926266+0000","last_became_active":"2024-04-25T01:09:44.942992+0000","last_became_peered":"2024-04-25T01:09:44.942992+0000","last_unstale":"2024-04-25T01:09:45.926266+0000","last_undegraded":"2024-04-25T01:09:45.926266+0000","last_fullsized":"2024-04-25T01:09:45.926266+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:32:01.020214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948807+0000","last_change":"2024-04-25T01:09:45.955997+0000","last_active":"2024-04-25T01:09:46.948807+0000","last_peered":"2024-04-25T01:09:46.948807+0000","last_clean":"2024-04-25T01:09:46.948807+0000","last_became_active":"2024-04-25T01:09:45.953312+0000","last_became_peered":"2024-04-25T01:09:45.953312+0000","last_unstale":"2024-04-25T01:09:46.948807+0000","last_undegraded":"2024-04-25T01:09:46.948807+0000","last_fullsized":"2024-04-25T01:09:46.948807+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:49.282873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.760941+0000","last_change":"2024-04-25T01:09:44.952579+0000","last_active":"2024-04-25T01:09:47.760941+0000","last_peered":"2024-04-25T01:09:47.760941+0000","last_clean":"2024-04-25T01:09:47.760941+0000","last_became_active":"2024-04-25T01:09:44.951862+0000","last_became_peered":"2024-04-25T01:09:44.951862+0000","last_unstale":"2024-04-25T01:09:47.760941+0000","last_undegraded":"2024-04-25T01:09:47.760941+0000","last_fullsized":"2024-04-25T01:09:47.760941+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:55.605009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948680+0000","last_change":"2024-04-25T01:09:45.957482+0000","last_active":"2024-04-25T01:09:46.948680+0000","last_peered":"2024-04-25T01:09:46.948680+0000","last_clean":"2024-04-25T01:09:46.948680+0000","last_became_active":"2024-04-25T01:09:45.956793+0000","last_became_peered":"2024-04-25T01:09:45.956793+0000","last_unstale":"2024-04-25T01:09:46.948680+0000","last_undegraded":"2024-04-25T01:09:46.948680+0000","last_fullsized":"2024-04-25T01:09:46.948680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:51.065354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.996787+0000","last_change":"2024-04-25T01:09:44.948130+0000","last_active":"2024-04-25T01:09:45.996787+0000","last_peered":"2024-04-25T01:09:45.996787+0000","last_clean":"2024-04-25T01:09:45.996787+0000","last_became_active":"2024-04-25T01:09:44.947424+0000","last_became_peered":"2024-04-25T01:09:44.947424+0000","last_unstale":"2024-04-25T01:09:45.996787+0000","last_undegraded":"2024-04-25T01:09:45.996787+0000","last_fullsized":"2024-04-25T01:09:45.996787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:58.363142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992541+0000","last_change":"2024-04-25T01:09:45.988900+0000","last_active":"2024-04-25T01:09:45.992541+0000","last_peered":"2024-04-25T01:09:45.992541+0000","last_clean":"2024-04-25T01:09:45.992541+0000","last_became_active":"2024-04-25T01:09:45.987712+0000","last_became_peered":"2024-04-25T01:09:45.987712+0000","last_unstale":"2024-04-25T01:09:45.992541+0000","last_undegraded":"2024-04-25T01:09:45.992541+0000","last_fullsized":"2024-04-25T01:09:45.992541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:49.504705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949997+0000","last_change":"2024-04-25T01:09:44.951214+0000","last_active":"2024-04-25T01:09:46.949997+0000","last_peered":"2024-04-25T01:09:46.949997+0000","last_clean":"2024-04-25T01:09:46.949997+0000","last_became_active":"2024-04-25T01:09:44.950454+0000","last_became_peered":"2024-04-25T01:09:44.950454+0000","last_unstale":"2024-04-25T01:09:46.949997+0000","last_undegraded":"2024-04-25T01:09:46.949997+0000","last_fullsized":"2024-04-25T01:09:46.949997+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:30.503466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949699+0000","last_change":"2024-04-25T01:09:45.970970+0000","last_active":"2024-04-25T01:09:46.949699+0000","last_peered":"2024-04-25T01:09:46.949699+0000","last_clean":"2024-04-25T01:09:46.949699+0000","last_became_active":"2024-04-25T01:09:45.970249+0000","last_became_peered":"2024-04-25T01:09:45.970249+0000","last_unstale":"2024-04-25T01:09:46.949699+0000","last_undegraded":"2024-04-25T01:09:46.949699+0000","last_fullsized":"2024-04-25T01:09:46.949699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:03:04.164495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958351+0000","last_change":"2024-04-25T01:09:44.940187+0000","last_active":"2024-04-25T01:09:46.958351+0000","last_peered":"2024-04-25T01:09:46.958351+0000","last_clean":"2024-04-25T01:09:46.958351+0000","last_became_active":"2024-04-25T01:09:44.939828+0000","last_became_peered":"2024-04-25T01:09:44.939828+0000","last_unstale":"2024-04-25T01:09:46.958351+0000","last_undegraded":"2024-04-25T01:09:46.958351+0000","last_fullsized":"2024-04-25T01:09:46.958351+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:16:44.190014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958791+0000","last_change":"2024-04-25T01:09:45.951217+0000","last_active":"2024-04-25T01:09:46.958791+0000","last_peered":"2024-04-25T01:09:46.958791+0000","last_clean":"2024-04-25T01:09:46.958791+0000","last_became_active":"2024-04-25T01:09:45.950924+0000","last_became_peered":"2024-04-25T01:09:45.950924+0000","last_unstale":"2024-04-25T01:09:46.958791+0000","last_undegraded":"2024-04-25T01:09:46.958791+0000","last_fullsized":"2024-04-25T01:09:46.958791+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:25.863316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.938735+0000","last_change":"2024-04-25T01:09:44.934181+0000","last_active":"2024-04-25T01:09:45.938735+0000","last_peered":"2024-04-25T01:09:45.938735+0000","last_clean":"2024-04-25T01:09:45.938735+0000","last_became_active":"2024-04-25T01:09:44.933868+0000","last_became_peered":"2024-04-25T01:09:44.933868+0000","last_unstale":"2024-04-25T01:09:45.938735+0000","last_undegraded":"2024-04-25T01:09:45.938735+0000","last_fullsized":"2024-04-25T01:09:45.938735+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:55:24.853029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761952+0000","last_change":"2024-04-25T01:09:45.953111+0000","last_active":"2024-04-25T01:09:47.761952+0000","last_peered":"2024-04-25T01:09:47.761952+0000","last_clean":"2024-04-25T01:09:47.761952+0000","last_became_active":"2024-04-25T01:09:45.952331+0000","last_became_peered":"2024-04-25T01:09:45.952331+0000","last_unstale":"2024-04-25T01:09:47.761952+0000","last_undegraded":"2024-04-25T01:09:47.761952+0000","last_fullsized":"2024-04-25T01:09:47.761952+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:34.776822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.958339+0000","last_change":"2024-04-25T01:09:44.926162+0000","last_active":"2024-04-25T01:09:46.958339+0000","last_peered":"2024-04-25T01:09:46.958339+0000","last_clean":"2024-04-25T01:09:46.958339+0000","last_became_active":"2024-04-25T01:09:44.925425+0000","last_became_peered":"2024-04-25T01:09:44.925425+0000","last_unstale":"2024-04-25T01:09:46.958339+0000","last_undegraded":"2024-04-25T01:09:46.958339+0000","last_fullsized":"2024-04-25T01:09:46.958339+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:17.588543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.991371+0000","last_change":"2024-04-25T01:09:45.989908+0000","last_active":"2024-04-25T01:09:45.991371+0000","last_peered":"2024-04-25T01:09:45.991371+0000","last_clean":"2024-04-25T01:09:45.991371+0000","last_became_active":"2024-04-25T01:09:45.989587+0000","last_became_peered":"2024-04-25T01:09:45.989587+0000","last_unstale":"2024-04-25T01:09:45.991371+0000","last_undegraded":"2024-04-25T01:09:45.991371+0000","last_fullsized":"2024-04-25T01:09:45.991371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:30.479872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.957923+0000","last_change":"2024-04-25T01:09:44.923407+0000","last_active":"2024-04-25T01:09:46.957923+0000","last_peered":"2024-04-25T01:09:46.957923+0000","last_clean":"2024-04-25T01:09:46.957923+0000","last_became_active":"2024-04-25T01:09:44.923041+0000","last_became_peered":"2024-04-25T01:09:44.923041+0000","last_unstale":"2024-04-25T01:09:46.957923+0000","last_undegraded":"2024-04-25T01:09:46.957923+0000","last_fullsized":"2024-04-25T01:09:46.957923+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:23.364045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.989715+0000","last_change":"2024-04-25T01:09:45.974584+0000","last_active":"2024-04-25T01:09:45.989715+0000","last_peered":"2024-04-25T01:09:45.989715+0000","last_clean":"2024-04-25T01:09:45.989715+0000","last_became_active":"2024-04-25T01:09:45.974075+0000","last_became_peered":"2024-04-25T01:09:45.974075+0000","last_unstale":"2024-04-25T01:09:45.989715+0000","last_undegraded":"2024-04-25T01:09:45.989715+0000","last_fullsized":"2024-04-25T01:09:45.989715+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:31:30.479872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.762019+0000","last_change":"2024-04-25T01:09:44.949067+0000","last_active":"2024-04-25T01:09:47.762019+0000","last_peered":"2024-04-25T01:09:47.762019+0000","last_clean":"2024-04-25T01:09:47.762019+0000","last_became_active":"2024-04-25T01:09:44.948285+0000","last_became_peered":"2024-04-25T01:09:44.948285+0000","last_unstale":"2024-04-25T01:09:47.762019+0000","last_undegraded":"2024-04-25T01:09:47.762019+0000","last_fullsized":"2024-04-25T01:09:47.762019+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:42.725733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948832+0000","last_change":"2024-04-25T01:09:45.990591+0000","last_active":"2024-04-25T01:09:46.948832+0000","last_peered":"2024-04-25T01:09:46.948832+0000","last_clean":"2024-04-25T01:09:46.948832+0000","last_became_active":"2024-04-25T01:09:45.989676+0000","last_became_peered":"2024-04-25T01:09:45.989676+0000","last_unstale":"2024-04-25T01:09:46.948832+0000","last_undegraded":"2024-04-25T01:09:46.948832+0000","last_fullsized":"2024-04-25T01:09:46.948832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:37.680178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.957558+0000","last_change":"2024-04-25T01:09:44.931330+0000","last_active":"2024-04-25T01:09:46.957558+0000","last_peered":"2024-04-25T01:09:46.957558+0000","last_clean":"2024-04-25T01:09:46.957558+0000","last_became_active":"2024-04-25T01:09:44.930909+0000","last_became_peered":"2024-04-25T01:09:44.930909+0000","last_unstale":"2024-04-25T01:09:46.957558+0000","last_undegraded":"2024-04-25T01:09:46.957558+0000","last_fullsized":"2024-04-25T01:09:46.957558+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:38:14.765826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.980760+0000","last_change":"2024-04-25T01:09:45.976207+0000","last_active":"2024-04-25T01:09:45.980760+0000","last_peered":"2024-04-25T01:09:45.980760+0000","last_clean":"2024-04-25T01:09:45.980760+0000","last_became_active":"2024-04-25T01:09:45.975900+0000","last_became_peered":"2024-04-25T01:09:45.975900+0000","last_unstale":"2024-04-25T01:09:45.980760+0000","last_undegraded":"2024-04-25T01:09:45.980760+0000","last_fullsized":"2024-04-25T01:09:45.980760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:10.492363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992577+0000","last_change":"2024-04-25T01:09:44.957399+0000","last_active":"2024-04-25T01:09:45.992577+0000","last_peered":"2024-04-25T01:09:45.992577+0000","last_clean":"2024-04-25T01:09:45.992577+0000","last_became_active":"2024-04-25T01:09:44.956692+0000","last_became_peered":"2024-04-25T01:09:44.956692+0000","last_unstale":"2024-04-25T01:09:45.992577+0000","last_undegraded":"2024-04-25T01:09:45.992577+0000","last_fullsized":"2024-04-25T01:09:45.992577+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:13.312268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761930+0000","last_change":"2024-04-25T01:09:45.972444+0000","last_active":"2024-04-25T01:09:47.761930+0000","last_peered":"2024-04-25T01:09:47.761930+0000","last_clean":"2024-04-25T01:09:47.761930+0000","last_became_active":"2024-04-25T01:09:45.965392+0000","last_became_peered":"2024-04-25T01:09:45.965392+0000","last_unstale":"2024-04-25T01:09:47.761930+0000","last_undegraded":"2024-04-25T01:09:47.761930+0000","last_fullsized":"2024-04-25T01:09:47.761930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:53:47.627797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.961177+0000","last_change":"2024-04-25T01:09:44.939689+0000","last_active":"2024-04-25T01:09:46.961177+0000","last_peered":"2024-04-25T01:09:46.961177+0000","last_clean":"2024-04-25T01:09:46.961177+0000","last_became_active":"2024-04-25T01:09:44.939276+0000","last_became_peered":"2024-04-25T01:09:44.939276+0000","last_unstale":"2024-04-25T01:09:46.961177+0000","last_undegraded":"2024-04-25T01:09:46.961177+0000","last_fullsized":"2024-04-25T01:09:46.961177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:08.749343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.948726+0000","last_change":"2024-04-25T01:09:45.979277+0000","last_active":"2024-04-25T01:09:46.948726+0000","last_peered":"2024-04-25T01:09:46.948726+0000","last_clean":"2024-04-25T01:09:46.948726+0000","last_became_active":"2024-04-25T01:09:45.978969+0000","last_became_peered":"2024-04-25T01:09:45.978969+0000","last_unstale":"2024-04-25T01:09:46.948726+0000","last_undegraded":"2024-04-25T01:09:46.948726+0000","last_fullsized":"2024-04-25T01:09:46.948726+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:56.733946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.934867+0000","last_change":"2024-04-25T01:09:44.928526+0000","last_active":"2024-04-25T01:09:45.934867+0000","last_peered":"2024-04-25T01:09:45.934867+0000","last_clean":"2024-04-25T01:09:45.934867+0000","last_became_active":"2024-04-25T01:09:44.927791+0000","last_became_peered":"2024-04-25T01:09:44.927791+0000","last_unstale":"2024-04-25T01:09:45.934867+0000","last_undegraded":"2024-04-25T01:09:45.934867+0000","last_fullsized":"2024-04-25T01:09:45.934867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:42:56.941785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.948190+0000","last_change":"2024-04-25T01:09:45.946961+0000","last_active":"2024-04-25T01:09:45.948190+0000","last_peered":"2024-04-25T01:09:45.948190+0000","last_clean":"2024-04-25T01:09:45.948190+0000","last_became_active":"2024-04-25T01:09:45.946659+0000","last_became_peered":"2024-04-25T01:09:45.946659+0000","last_unstale":"2024-04-25T01:09:45.948190+0000","last_undegraded":"2024-04-25T01:09:45.948190+0000","last_fullsized":"2024-04-25T01:09:45.948190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:14.322556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.935279+0000","last_change":"2024-04-25T01:09:44.935837+0000","last_active":"2024-04-25T01:09:45.935279+0000","last_peered":"2024-04-25T01:09:45.935279+0000","last_clean":"2024-04-25T01:09:45.935279+0000","last_became_active":"2024-04-25T01:09:44.935376+0000","last_became_peered":"2024-04-25T01:09:44.935376+0000","last_unstale":"2024-04-25T01:09:45.935279+0000","last_undegraded":"2024-04-25T01:09:45.935279+0000","last_fullsized":"2024-04-25T01:09:45.935279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:37:54.518392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.948645+0000","last_change":"2024-04-25T01:09:45.945819+0000","last_active":"2024-04-25T01:09:45.948645+0000","last_peered":"2024-04-25T01:09:45.948645+0000","last_clean":"2024-04-25T01:09:45.948645+0000","last_became_active":"2024-04-25T01:09:45.945436+0000","last_became_peered":"2024-04-25T01:09:45.945436+0000","last_unstale":"2024-04-25T01:09:45.948645+0000","last_undegraded":"2024-04-25T01:09:45.948645+0000","last_fullsized":"2024-04-25T01:09:45.948645+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:01.017792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.927269+0000","last_change":"2024-04-25T01:09:44.937941+0000","last_active":"2024-04-25T01:09:45.927269+0000","last_peered":"2024-04-25T01:09:45.927269+0000","last_clean":"2024-04-25T01:09:45.927269+0000","last_became_active":"2024-04-25T01:09:44.937143+0000","last_became_peered":"2024-04-25T01:09:44.937143+0000","last_unstale":"2024-04-25T01:09:45.927269+0000","last_undegraded":"2024-04-25T01:09:45.927269+0000","last_fullsized":"2024-04-25T01:09:45.927269+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:21:04.012622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949442+0000","last_change":"2024-04-25T01:09:45.975669+0000","last_active":"2024-04-25T01:09:46.949442+0000","last_peered":"2024-04-25T01:09:46.949442+0000","last_clean":"2024-04-25T01:09:46.949442+0000","last_became_active":"2024-04-25T01:09:45.971884+0000","last_became_peered":"2024-04-25T01:09:45.971884+0000","last_unstale":"2024-04-25T01:09:46.949442+0000","last_undegraded":"2024-04-25T01:09:46.949442+0000","last_fullsized":"2024-04-25T01:09:46.949442+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:43.284052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.949100+0000","last_change":"2024-04-25T01:09:44.923165+0000","last_active":"2024-04-25T01:09:46.949100+0000","last_peered":"2024-04-25T01:09:46.949100+0000","last_clean":"2024-04-25T01:09:46.949100+0000","last_became_active":"2024-04-25T01:09:44.922477+0000","last_became_peered":"2024-04-25T01:09:44.922477+0000","last_unstale":"2024-04-25T01:09:46.949100+0000","last_undegraded":"2024-04-25T01:09:46.949100+0000","last_fullsized":"2024-04-25T01:09:46.949100+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:01:28.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761850+0000","last_change":"2024-04-25T01:09:45.971665+0000","last_active":"2024-04-25T01:09:47.761850+0000","last_peered":"2024-04-25T01:09:47.761850+0000","last_clean":"2024-04-25T01:09:47.761850+0000","last_became_active":"2024-04-25T01:09:45.970827+0000","last_became_peered":"2024-04-25T01:09:45.970827+0000","last_unstale":"2024-04-25T01:09:47.761850+0000","last_undegraded":"2024-04-25T01:09:47.761850+0000","last_fullsized":"2024-04-25T01:09:47.761850+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:49.670969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:46.947916+0000","last_change":"2024-04-25T01:09:44.928414+0000","last_active":"2024-04-25T01:09:46.947916+0000","last_peered":"2024-04-25T01:09:46.947916+0000","last_clean":"2024-04-25T01:09:46.947916+0000","last_became_active":"2024-04-25T01:09:44.927870+0000","last_became_peered":"2024-04-25T01:09:44.927870+0000","last_unstale":"2024-04-25T01:09:46.947916+0000","last_undegraded":"2024-04-25T01:09:46.947916+0000","last_fullsized":"2024-04-25T01:09:46.947916+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:33.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T01:09:45.992446+0000","last_change":"2024-04-25T01:09:45.976053+0000","last_active":"2024-04-25T01:09:45.992446+0000","last_peered":"2024-04-25T01:09:45.992446+0000","last_clean":"2024-04-25T01:09:45.992446+0000","last_became_active":"2024-04-25T01:09:45.960272+0000","last_became_peered":"2024-04-25T01:09:45.960272+0000","last_unstale":"2024-04-25T01:09:45.992446+0000","last_undegraded":"2024-04-25T01:09:45.992446+0000","last_fullsized":"2024-04-25T01:09:45.992446+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:44.898929+0000","last_clean_scrub_stamp":"2024-04-25T01:09:44.898929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:20:15.611671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761937+0000","last_change":"2024-04-25T01:09:44.943334+0000","last_active":"2024-04-25T01:09:47.761937+0000","last_peered":"2024-04-25T01:09:47.761937+0000","last_clean":"2024-04-25T01:09:47.761937+0000","last_became_active":"2024-04-25T01:09:44.942647+0000","last_became_peered":"2024-04-25T01:09:44.942647+0000","last_unstale":"2024-04-25T01:09:47.761937+0000","last_undegraded":"2024-04-25T01:09:47.761937+0000","last_fullsized":"2024-04-25T01:09:47.761937+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:09:43.888642+0000","last_clean_scrub_stamp":"2024-04-25T01:09:43.888642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:42:00.007505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:09:47.761258+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:09:47.761258+0000","last_peered":"2024-04-25T01:09:47.761258+0000","last_clean":"2024-04-25T01:09:47.761258+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:09:47.761258+0000","last_undegraded":"2024-04-25T01:09:47.761258+0000","last_fullsized":"2024-04-25T01:09:47.761258+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1052672,"data_stored":784285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":186125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":192839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":458752,"data_stored":193881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":192305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":185633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":417792,"data_stored":178376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574869,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":768744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:09:50.384 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T01:09:50.384 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T01:09:50.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.556+0000 7f95afa7b640 1 Processor -- start 2024-04-25T01:09:50.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.556+0000 7f95afa7b640 1 -- start start 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.556+0000 7f95afa7b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95a8145660 0x7f95a806eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.556+0000 7f95afa7b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 0x7f95a815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95afa7b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a8160770 con 0x7f95a81602e0 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95afa7b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f95a806f0a0 con 0x7f95a8145660 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95afa7b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95a8145a40 con 0x7f95a815f960 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ae018640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f95a81602e0 0x7f95a806d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53514/0 (socket says 172.21.15.33:53514) 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ae018640 1 -- 172.21.15.33:0/2517131005 learned_addr learned my addr 172.21.15.33:0/2517131005 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:50.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 --2- 172.21.15.33:0/2517131005 >> v2:172.21.15.138:3300/0 conn(0x7f95a8145660 0x7f95a806eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad016640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 0x7f95a815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3776045263 0 0) 0x7f95a8160770 con 0x7f95a81602e0 2024-04-25T01:09:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9594003530 con 0x7f95a81602e0 2024-04-25T01:09:50.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303730125 0 0) 0x7f9594003530 con 0x7f95a81602e0 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 msgr2=0x7f95a815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 0x7f95a815fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 >> v2:172.21.15.138:3300/0 conn(0x7f95a8145660 msgr2=0x7f95a806eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 --2- 172.21.15.33:0/2517131005 >> v2:172.21.15.138:3300/0 conn(0x7f95a8145660 0x7f95a806eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a80cee10 con 0x7f95a81602e0 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f95a0001a00 con 0x7f95a81602e0 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 >> v1:172.21.15.33:6789/0 conn(0x7f95a81602e0 legacy=0x7f95a806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95940039c0 0x7f9594003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 0x7f9594004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9594004d50 0x7f95940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95a806f0a0 con 0x7f95940039c0 2024-04-25T01:09:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f959400b9d0 con 0x7f9594004d50 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f959400bb50 con 0x7f95a815f960 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95940039c0 0x7f9594003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad016640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 0x7f9594004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 -- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 msgr2=0x7f9594004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95a815f960 0x7f9594004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ae018640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9594004d50 0x7f95940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 -- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9594004d50 msgr2=0x7f95940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9594004d50 0x7f95940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 -- 172.21.15.33:0/2517131005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9594003530 con 0x7f95940039c0 2024-04-25T01:09:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ae018640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9594004d50 0x7f95940051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ad817640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95940039c0 0x7f9594003de0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009ae0 tx=0x7f95a401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=910b9b83ddf1e07b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40b2070 con 0x7f95940039c0 2024-04-25T01:09:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95a40a5570 con 0x7f95940039c0 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.560+0000 7f95ac815640 1 -- 172.21.15.33:0/2517131005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a40096b0 con 0x7f95940039c0 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- 172.21.15.33:0/2517131005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95940039c0 msgr2=0x7f9594003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 --2- 172.21.15.33:0/2517131005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95940039c0 0x7f9594003de0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009ae0 tx=0x7f95a401cab0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ae018640 1 -- 172.21.15.33:0/2517131005 reap_dead start 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- 172.21.15.33:0/2517131005 shutdown_connections 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- 172.21.15.33:0/2517131005 >> 172.21.15.33:0/2517131005 conn(0x7f95a80b1810 msgr2=0x7f95a80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- 172.21.15.33:0/2517131005 shutdown_connections 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- 172.21.15.33:0/2517131005 wait complete. 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 Processor -- start 2024-04-25T01:09:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- start start 2024-04-25T01:09:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940039c0 0x7f95a813f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95a8143fe0 0x7f95a8144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a80ad990 0x7f95a80ade10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8166c90 con 0x7f95a8143fe0 2024-04-25T01:09:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95a8166e10 con 0x7f95a80ad990 2024-04-25T01:09:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f95a8166f90 con 0x7f95940039c0 2024-04-25T01:09:50.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95a8143fe0 0x7f95a8144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95a8143fe0 0x7f95a8144460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43070/0 (socket says 172.21.15.33:43070) 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 -- 172.21.15.33:0/1516237187 learned_addr learned my addr 172.21.15.33:0/1516237187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad817640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940039c0 0x7f95a813f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ae018640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a80ad990 0x7f95a80ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 -- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940039c0 msgr2=0x7f95a813f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940039c0 0x7f95a813f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 -- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a80ad990 msgr2=0x7f95a80ade10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a80ad990 0x7f95a80ade10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 -- 172.21.15.33:0/1516237187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a80b2cb0 con 0x7f95a8143fe0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95ad016640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95a8143fe0 0x7f95a8144460 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f959c011610 tx=0x7f959c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f959a7fc640 1 -- 172.21.15.33:0/1516237187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00f8f0 con 0x7f95a8143fe0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f959a7fc640 1 -- 172.21.15.33:0/1516237187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c00fa90 con 0x7f95a8143fe0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a80b2f90 con 0x7f95a8143fe0 2024-04-25T01:09:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f959a7fc640 1 -- 172.21.15.33:0/1516237187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00fe40 con 0x7f95a8143fe0 2024-04-25T01:09:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.564+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a80b3550 con 0x7f95a8143fe0 2024-04-25T01:09:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.568+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a8069040 con 0x7f95a8143fe0 2024-04-25T01:09:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.568+0000 7f959a7fc640 1 -- 172.21.15.33:0/1516237187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f959c015460 con 0x7f95a8143fe0 2024-04-25T01:09:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.568+0000 7f959a7fc640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f958c0678c0 0x7f958c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.568+0000 7f959a7fc640 1 -- 172.21.15.33:0/1516237187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f959c0867b0 con 0x7f95a8143fe0 2024-04-25T01:09:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.568+0000 7f95ad817640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f958c0678c0 0x7f958c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.568+0000 7f95ad817640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f958c0678c0 0x7f958c069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f95a40af000 tx=0x7f95a40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.572+0000 7f959a7fc640 1 -- 172.21.15.33:0/1516237187 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f959c019030 con 0x7f95a8143fe0 2024-04-25T01:09:50.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.784+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f95a8162a50 con 0x7f95a8143fe0 2024-04-25T01:09:50.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f959a7fc640 1 -- 172.21.15.33:0/1516237187 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f959c057240 con 0x7f95a8143fe0 2024-04-25T01:09:50.779 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-04-25T01:09:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDUrClmzPTbIhAAvzORiMqrE/Y+uTLvKO5BMw== 2024-04-25T01:09:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: caps mds = "allow" 2024-04-25T01:09:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: caps mon = "allow r" 2024-04-25T01:09:50.779 INFO:teuthology.orchestra.run.smithi033.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T01:09:50.780 INFO:teuthology.orchestra.run.smithi033.stderr:updated caps for client.0 2024-04-25T01:09:50.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f958c0678c0 msgr2=0x7f958c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f958c0678c0 0x7f958c069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f95a40af000 tx=0x7f95a40046d0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95a8143fe0 msgr2=0x7f95a8144460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95a8143fe0 0x7f95a8144460 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f959c011610 tx=0x7f959c011ad0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 shutdown_connections 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f958c0678c0 0x7f958c069d80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a80ad990 0x7f95a80ade10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95a8143fe0 0x7f95a8144460 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 --2- 172.21.15.33:0/1516237187 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f95940039c0 0x7f95a813f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 >> 172.21.15.33:0/1516237187 conn(0x7f95a80b1810 msgr2=0x7f95a8140300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 shutdown_connections 2024-04-25T01:09:50.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.788+0000 7f95afa7b640 1 -- 172.21.15.33:0/1516237187 wait complete. 2024-04-25T01:09:50.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:09:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.912+0000 7fc90151a640 1 Processor -- start 2024-04-25T01:09:50.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.912+0000 7fc90151a640 1 -- start start 2024-04-25T01:09:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc90151a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc8fc0d03e0 0x7fc8fc0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc90151a640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc8fc0d48d0 0x7fc8fc0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc90151a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8fc0d10b0 con 0x7fc8fc15e470 2024-04-25T01:09:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc90151a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fc8fc0cff10 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc90151a640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc8fc0d1de0 con 0x7fc8fc0d48d0 2024-04-25T01:09:50.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8faffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fc8fc0d03e0 0x7fc8fc0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8fa7fc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc8fc15e470 0x7fc8fc0d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53530/0 (socket says 172.21.15.33:53530) 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8fa7fc640 1 -- 172.21.15.33:0/983174476 learned_addr learned my addr 172.21.15.33:0/983174476 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8fb7fe640 1 --2- 172.21.15.33:0/983174476 >> v2:172.21.15.138:3300/0 conn(0x7fc8fc0d48d0 0x7fc8fc0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8f9ffb640 1 -- 172.21.15.33:0/983174476 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3673142580 0 0) 0x7fc8fc0d10b0 con 0x7fc8fc15e470 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8f9ffb640 1 -- 172.21.15.33:0/983174476 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8e4003530 con 0x7fc8fc15e470 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8faffd640 1 -- 172.21.15.33:0/983174476 >> v2:172.21.15.138:3300/0 conn(0x7fc8fc0d48d0 msgr2=0x7fc8fc0cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8faffd640 1 --2- 172.21.15.33:0/983174476 >> v2:172.21.15.138:3300/0 conn(0x7fc8fc0d48d0 0x7fc8fc0cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8faffd640 1 -- 172.21.15.33:0/983174476 >> v1:172.21.15.33:6789/0 conn(0x7fc8fc15e470 legacy=0x7fc8fc0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8faffd640 1 -- 172.21.15.33:0/983174476 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc0d1390 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8faffd640 1 --2- 172.21.15.33:0/983174476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc0d07c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc08e0c0 tx=0x7fc8ec01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c12ecfdccbed9df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8f9ffb640 1 -- 172.21.15.33:0/983174476 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0b0070 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8f9ffb640 1 -- 172.21.15.33:0/983174476 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ec0ab600 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.916+0000 7fc8f9ffb640 1 -- 172.21.15.33:0/983174476 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0ab9b0 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/983174476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 msgr2=0x7fc8fc0d07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 --2- 172.21.15.33:0/983174476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc0d07c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc08e0c0 tx=0x7fc8ec01c390 comp rx=0 tx=0).stop 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/983174476 shutdown_connections 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 --2- 172.21.15.33:0/983174476 >> v2:172.21.15.138:3300/0 conn(0x7fc8fc0d48d0 0x7fc8fc0cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 --2- 172.21.15.33:0/983174476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc0d07c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/983174476 >> 172.21.15.33:0/983174476 conn(0x7fc8fc0d3db0 msgr2=0x7fc8fc0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/983174476 shutdown_connections 2024-04-25T01:09:50.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/983174476 wait complete. 2024-04-25T01:09:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 Processor -- start 2024-04-25T01:09:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- start start 2024-04-25T01:09:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8fc0d48d0 0x7fc8fc158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8fc15e470 0x7fc8fc156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc0d3310 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8fc0d3190 con 0x7fc8fc0d48d0 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc8fc0d3490 con 0x7fc8fc15e470 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43090/0 (socket says 172.21.15.33:43090) 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 -- 172.21.15.33:0/3103322176 learned_addr learned my addr 172.21.15.33:0/3103322176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8fb7fe640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8fc15e470 0x7fc8fc156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 -- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8fc15e470 msgr2=0x7fc8fc156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8fc15e470 0x7fc8fc156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8fa7fc640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8fc0d48d0 0x7fc8fc158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 -- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8fc0d48d0 msgr2=0x7fc8fc158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8fc0d48d0 0x7fc8fc158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 -- 172.21.15.33:0/3103322176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8fc156080 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8faffd640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc155b40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec000c10 tx=0x7fc8ec019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8e37fe640 1 -- 172.21.15.33:0/3103322176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0ba070 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8e37fe640 1 -- 172.21.15.33:0/3103322176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8ec0ab9b0 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc8e37fe640 1 -- 172.21.15.33:0/3103322176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec0b0040 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8fc15b0c0 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8fc15b680 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.920+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8fc069040 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.924+0000 7fc8e37fe640 1 -- 172.21.15.33:0/3103322176 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8ec0ab480 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.924+0000 7fc8e37fe640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:50.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.924+0000 7fc8e37fe640 1 -- 172.21.15.33:0/3103322176 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc8ec11ef80 con 0x7fc8fc0d03e0 2024-04-25T01:09:50.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.924+0000 7fc8fa7fc640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:50.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.924+0000 7fc8fa7fc640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc8e80678c0 0x7fc8e8069d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0007fd0 tx=0x7fc8f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:50.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:50.928+0000 7fc8e37fe640 1 -- 172.21.15.33:0/3103322176 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc8fc156080 con 0x7fc8fc0d03e0 2024-04-25T01:09:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.116+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8fc0d3b90 con 0x7fc8fc0d03e0 2024-04-25T01:09:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc8e37fe640 1 -- 172.21.15.33:0/3103322176 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fc8ec016070 con 0x7fc8fc0d03e0 2024-04-25T01:09:51.111 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:09:51.111 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3228956110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3228956110},{"type":"v1","addr":"172.21.15.138:6835","nonce":3228956110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2116493369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2116493369},{"type":"v1","addr":"172.21.15.33:6835","nonce":2116493369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2610873006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2610873006},{"type":"v1","addr":"172.21.15.138:6837","nonce":2610873006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:09:45.288708+0000","modified":"2024-04-25T01:09:48.318253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/3726672082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3726672082},{"type":"v1","addr":"172.21.15.33:6834","nonce":3726672082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-25T01:09:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-25T01:09:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc8e80678c0 msgr2=0x7fc8e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc8e80678c0 0x7fc8e8069d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0007fd0 tx=0x7fc8f0009040 comp rx=0 tx=0).stop 2024-04-25T01:09:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 msgr2=0x7fc8fc155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc155b40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec000c10 tx=0x7fc8ec019190 comp rx=0 tx=0).stop 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 shutdown_connections 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc8e80678c0 0x7fc8e8069d80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc8fc15e470 0x7fc8fc156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8fc0d48d0 0x7fc8fc158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 --2- 172.21.15.33:0/3103322176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc8fc0d03e0 0x7fc8fc155b40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 >> 172.21.15.33:0/3103322176 conn(0x7fc8fc0d3db0 msgr2=0x7fc8fc0b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 shutdown_connections 2024-04-25T01:09:51.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.120+0000 7fc90151a640 1 -- 172.21.15.33:0/3103322176 wait complete. 2024-04-25T01:09:51.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T01:09:45.288708+0000', 'modified': '2024-04-25T01:09:48.318253+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5520}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5520': {'gid': 5520, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.33:6834/3726672082', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6832', 'nonce': 3726672082}, {'type': 'v1', 'addr': '172.21.15.33:6834', 'nonce': 3726672082}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5520, 'qdb_cluster': [5520]} 2024-04-25T01:09:51.532 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T01:09:51.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T01:09:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.672+0000 7f375af14640 1 Processor -- start 2024-04-25T01:09:51.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.672+0000 7f375af14640 1 -- start start 2024-04-25T01:09:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f375af14640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f37540b02c0 0x7f375415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f375af14640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37540af540 con 0x7f375415ebf0 2024-04-25T01:09:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f375af14640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37540aece0 con 0x7f375415ac50 2024-04-25T01:09:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f375af14640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f37540aef80 con 0x7f37540b02c0 2024-04-25T01:09:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37594b1640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f375415ebf0 0x7f37540b42f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:53544/0 (socket says 172.21.15.33:53544) 2024-04-25T01:09:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37594b1640 1 -- 172.21.15.33:0/933098718 learned_addr learned my addr 172.21.15.33:0/933098718 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:09:51.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3753fff640 1 --2- 172.21.15.33:0/933098718 >> v2:172.21.15.138:3300/0 conn(0x7f37540b02c0 0x7f375415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1159738238 0 0) 0x7f37540af540 con 0x7f375415ebf0 2024-04-25T01:09:51.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f373c005740 con 0x7f375415ebf0 2024-04-25T01:09:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 607997438 0 0) 0x7f373c005740 con 0x7f375415ebf0 2024-04-25T01:09:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 >> v2:172.21.15.138:3300/0 conn(0x7f37540b02c0 msgr2=0x7f375415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 --2- 172.21.15.33:0/933098718 >> v2:172.21.15.138:3300/0 conn(0x7f37540b02c0 0x7f375415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 >> v1:172.21.15.138:6789/0 conn(0x7f375415ac50 legacy=0x7f375406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37540af260 con 0x7f375415ebf0 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f374c001a00 con 0x7f375415ebf0 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 >> v1:172.21.15.33:6789/0 conn(0x7f375415ebf0 legacy=0x7f37540b42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c005bd0 0x7f373c005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f373c0065d0 0x7f373c006aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c007340 0x7f373c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37540aef80 con 0x7f373c005bd0 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f373c00e280 con 0x7f373c0065d0 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f373c00e480 con 0x7f373c007340 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3758cb0640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c005bd0 0x7f373c005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3753fff640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f373c0065d0 0x7f373c006aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37594b1640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c007340 0x7f373c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3758cb0640 1 -- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c007340 msgr2=0x7f373c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3758cb0640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c007340 0x7f373c00dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3758cb0640 1 -- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f373c0065d0 msgr2=0x7f373c006aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3758cb0640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f373c0065d0 0x7f373c006aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3758cb0640 1 -- 172.21.15.33:0/933098718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f373c005740 con 0x7f373c005bd0 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37594b1640 1 -- 172.21.15.33:0/933098718 reap_dead start 2024-04-25T01:09:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3753fff640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f373c0065d0 0x7f373c006aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f3758cb0640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c005bd0 0x7f373c005fb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3744001860 tx=0x7f374401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184eee5c6aca96d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37440b0070 con 0x7f373c005bd0 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37440ab600 con 0x7f373c005bd0 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.676+0000 7f37537fe640 1 -- 172.21.15.33:0/933098718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744001460 con 0x7f373c005bd0 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- 172.21.15.33:0/933098718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c005bd0 msgr2=0x7f373c005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c005bd0 0x7f373c005fb0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f3744001860 tx=0x7f374401c500 comp rx=0 tx=0).stop 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- 172.21.15.33:0/933098718 shutdown_connections 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 --2- 172.21.15.33:0/933098718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f373c005bd0 0x7f373c005fb0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- 172.21.15.33:0/933098718 >> 172.21.15.33:0/933098718 conn(0x7f37540b29c0 msgr2=0x7f37540b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- 172.21.15.33:0/933098718 shutdown_connections 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- 172.21.15.33:0/933098718 wait complete. 2024-04-25T01:09:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 Processor -- start 2024-04-25T01:09:51.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- start start 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c005bd0 0x7f375414c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f375415d870 0x7f3754071e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415dcd0 0x7f3754072350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f375406e040 con 0x7f375415dcd0 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f375406dec0 con 0x7f375415d870 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f375af14640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f375406e1c0 con 0x7f373c005bd0 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f37594b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415dcd0 0x7f3754072350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f37594b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415dcd0 0x7f3754072350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43098/0 (socket says 172.21.15.33:43098) 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f37594b1640 1 -- 172.21.15.33:0/1976824246 learned_addr learned my addr 172.21.15.33:0/1976824246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f3753fff640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f375415d870 0x7f3754071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.680+0000 7f3758cb0640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c005bd0 0x7f375414c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37594b1640 1 -- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c005bd0 msgr2=0x7f375414c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37594b1640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c005bd0 0x7f375414c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37594b1640 1 -- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f375415d870 msgr2=0x7f3754071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37594b1640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f375415d870 0x7f3754071e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37594b1640 1 -- 172.21.15.33:0/1976824246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37540b5130 con 0x7f375415dcd0 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f3753fff640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f375415d870 0x7f3754071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f3758cb0640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c005bd0 0x7f375414c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37594b1640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415dcd0 0x7f3754072350 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f374c000f90 tx=0x7f374c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37517fa640 1 -- 172.21.15.33:0/1976824246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c00bb80 con 0x7f375415dcd0 2024-04-25T01:09:51.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37517fa640 1 -- 172.21.15.33:0/1976824246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374c00bd20 con 0x7f375415dcd0 2024-04-25T01:09:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37541f3920 con 0x7f375415dcd0 2024-04-25T01:09:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37517fa640 1 -- 172.21.15.33:0/1976824246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c007290 con 0x7f375415dcd0 2024-04-25T01:09:51.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37541f3ee0 con 0x7f375415dcd0 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37517fa640 1 -- 172.21.15.33:0/1976824246 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f374c01b020 con 0x7f375415dcd0 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37517fa640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f37400678f0 0x7f3740069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f37517fa640 1 -- 172.21.15.33:0/1976824246 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f374c085fa0 con 0x7f375415dcd0 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f3758cb0640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f37400678f0 0x7f3740069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f375e08d000 1 Processor -- start 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.684+0000 7f375e08d000 1 -- start start 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f37400678f0 0x7f3740069db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f37440ad000 tx=0x7f37440a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b7ef0 0x5609853b82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x56098536e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56098536ebc0 0x56098536efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5609851bc5e0 con 0x5609853b7ef0 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560985143c50 con 0x5609853b8810 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x5609851b85a0 con 0x56098536ebc0 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b7ef0 0x5609853b82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b7ef0 0x5609853b82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43102/0 (socket says 172.21.15.33:43102) 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 -- 172.21.15.33:0/708106190 learned_addr learned my addr 172.21.15.33:0/708106190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3753fff640 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x56098536e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 -- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56098536ebc0 msgr2=0x56098536efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56098536ebc0 0x56098536efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 -- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 msgr2=0x56098536e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x56098536e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 -- 172.21.15.33:0/708106190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560985141d10 con 0x5609853b7ef0 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3753fff640 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x56098536e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b7ef0 0x5609853b82d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3744097e90 tx=0x7f3744001240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc25fc0652f4fb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f373a7fc640 1 -- 172.21.15.33:0/708106190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37440c6070 con 0x5609853b7ef0 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f373a7fc640 1 -- 172.21.15.33:0/708106190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37440075d0 con 0x5609853b7ef0 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f373a7fc640 1 -- 172.21.15.33:0/708106190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37440b0040 con 0x5609853b7ef0 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- 172.21.15.33:0/708106190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b7ef0 msgr2=0x5609853b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b7ef0 0x5609853b82d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3744097e90 tx=0x7f3744001240 comp rx=0 tx=0).stop 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- 172.21.15.33:0/708106190 shutdown_connections 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x56098536ebc0 0x56098536efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x56098536e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- 172.21.15.33:0/708106190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b7ef0 0x5609853b82d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- 172.21.15.33:0/708106190 >> 172.21.15.33:0/708106190 conn(0x56098536ab70 msgr2=0x56098536cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- 172.21.15.33:0/708106190 shutdown_connections 2024-04-25T01:09:51.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- 172.21.15.33:0/708106190 wait complete. 2024-04-25T01:09:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- start start 2024-04-25T01:09:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 Processor -- start 2024-04-25T01:09:51.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x56098539cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b7ef0 0x56098539d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b8810 0x56098539d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560985176520 con 0x56098536ebc0 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560985162810 con 0x5609853b7ef0 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x56098536a9f0 con 0x5609853b8810 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f375e08d000 10 client.0 _ll_register_callbacks cb 0x5609853595c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x56098539cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x56098539cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43110/0 (socket says 172.21.15.33:43110) 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.688+0000 7f3758cb0640 1 -- 172.21.15.33:0/713043549 learned_addr learned my addr 172.21.15.33:0/713043549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3753fff640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b7ef0 0x56098539d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37594b1640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b8810 0x56098539d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3758cb0640 1 -- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b8810 msgr2=0x56098539d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3758cb0640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b8810 0x56098539d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3758cb0640 1 -- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b7ef0 msgr2=0x56098539d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3758cb0640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b7ef0 0x56098539d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3758cb0640 1 -- 172.21.15.33:0/713043549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609851547b0 con 0x56098536ebc0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37594b1640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b8810 0x56098539d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3753fff640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b7ef0 0x56098539d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3758cb0640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x56098539cf20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3744001280 tx=0x7f37440192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbf3bb054d4ef76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 1 -- 172.21.15.33:0/713043549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37440c6070 con 0x56098536ebc0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37440075d0 con 0x56098536ebc0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 1 -- 172.21.15.33:0/713043549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f37440c1470 con 0x56098536ebc0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 1 -- 172.21.15.33:0/713043549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f37440b0040 con 0x56098536ebc0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 10 client.5718 fetch_fsmap learned FSMap version 17 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560985143ea0 con 0x56098536ebc0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 1 -- 172.21.15.33:0/713043549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37440075d0 con 0x56098536ebc0 2024-04-25T01:09:51.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 1 -- 172.21.15.33:0/713043549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f37440c1670 con 0x56098536ebc0 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 10 client.5718 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 10 client.5718 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 20 client.5718 populate_metadata read hostname 'smithi033' 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x5609853c7f90 0x5609853ca380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 4 client.5718 mds_command: new command op to 5520 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 --> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560985157ab0 con 0x5609853c7f90 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3753fff640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x5609853c7f90 0x5609853ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f3753fff640 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x5609853c7f90 0x5609853ca380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51db9b7f0a38dbf server_cookie=38aa3fc87fdac6e1 in_seq=0 out_seq=0 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 10 client.5718 ms_handle_connect on v2:172.21.15.33:6832/3726672082 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 1 -- 172.21.15.33:0/713043549 <== mds.0 v2:172.21.15.33:6832/3726672082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560985157ab0 con 0x5609853c7f90 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 10 client.5718 handle_command_reply: tid=0 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 client.5718 shutdown 2024-04-25T01:09:51.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 msgr2=0x56098539cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x56098539cf20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3744001280 tx=0x7f37440192a0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 shutdown_connections 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x5609853c7f90 0x5609853ca380 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b8810 0x56098539d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b7ef0 0x56098539d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 --2- 172.21.15.33:0/713043549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x56098539cf20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f37337fe640 0 client.5718 ms_handle_reset on v2:172.21.15.33:6832/3726672082 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 >> 172.21.15.33:0/713043549 conn(0x56098536ab70 msgr2=0x56098536d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 shutdown_connections 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 1 -- 172.21.15.33:0/713043549 wait complete. 2024-04-25T01:09:51.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.692+0000 7f375e08d000 20 client.5718 trim_cache size 0 max 16384 2024-04-25T01:09:51.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 Processor -- start 2024-04-25T01:09:51.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- start start 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x5609853c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 0x5609853c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x5609853c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f37541feb50 con 0x56098536ebc0 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f37541fdf00 con 0x5609853b8810 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f37541fe500 con 0x5609853b7ef0 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x5609853c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x5609853c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43114/0 (socket says 172.21.15.33:43114) 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 -- 172.21.15.33:0/1628578104 learned_addr learned my addr 172.21.15.33:0/1628578104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:51.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 -- 172.21.15.33:0/1628578104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 msgr2=0x5609853c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 --2- 172.21.15.33:0/1628578104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 0x5609853c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 -- 172.21.15.33:0/1628578104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 msgr2=0x5609853c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 --2- 172.21.15.33:0/1628578104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x5609853c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 -- 172.21.15.33:0/1628578104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560985144e80 con 0x56098536ebc0 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3758cb0640 1 --2- 172.21.15.33:0/1628578104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x5609853c8370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f374401c9e0 tx=0x7f37440c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4d319d4cb4b442 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3739ffb640 1 -- 172.21.15.33:0/1628578104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37440dd070 con 0x56098536ebc0 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3739ffb640 1 -- 172.21.15.33:0/1628578104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37440c8070 con 0x56098536ebc0 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f3739ffb640 1 -- 172.21.15.33:0/1628578104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37440c6070 con 0x56098536ebc0 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- 172.21.15.33:0/1628578104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 msgr2=0x5609853c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 --2- 172.21.15.33:0/1628578104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x5609853c8370 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f374401c9e0 tx=0x7f37440c3000 comp rx=0 tx=0).stop 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- 172.21.15.33:0/1628578104 shutdown_connections 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 --2- 172.21.15.33:0/1628578104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5609853b8810 0x5609853c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 --2- 172.21.15.33:0/1628578104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 0x5609853c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 --2- 172.21.15.33:0/1628578104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56098536ebc0 0x5609853c8370 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- 172.21.15.33:0/1628578104 >> 172.21.15.33:0/1628578104 conn(0x56098536ab70 msgr2=0x5609853c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- 172.21.15.33:0/1628578104 shutdown_connections 2024-04-25T01:09:51.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- 172.21.15.33:0/1628578104 wait complete. 2024-04-25T01:09:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 -- start start 2024-04-25T01:09:51.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.712+0000 7f375e08d000 1 Processor -- start 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56098536ebc0 0x560985427790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 0x560985427cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b8810 0x5609853ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56098526ee80 con 0x5609853b8810 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560985162810 con 0x56098536ebc0 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x560985170c40 con 0x5609853b7ef0 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 10 client.0 _ll_register_callbacks cb 0x7f37440c4070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b8810 0x5609853ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b8810 0x5609853ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43122/0 (socket says 172.21.15.33:43122) 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 -- 172.21.15.33:0/3133316942 learned_addr learned my addr 172.21.15.33:0/3133316942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 -- 172.21.15.33:0/3133316942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 msgr2=0x560985427cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 0x560985427cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 -- 172.21.15.33:0/3133316942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56098536ebc0 msgr2=0x560985427790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56098536ebc0 0x560985427790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 -- 172.21.15.33:0/3133316942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560985189760 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37594b1640 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b8810 0x5609853ca2d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f374c0012f0 tx=0x7f374c056a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f6fef785e11593 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 1 -- 172.21.15.33:0/3133316942 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c05a600 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 1 -- 172.21.15.33:0/3133316942 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f374c010d60 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37440c8070 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 1 -- 172.21.15.33:0/3133316942 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f374c05ee30 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 10 client.5724 fetch_fsmap learned FSMap version 17 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5609852159e0 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 1 -- 172.21.15.33:0/3133316942 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37440c8070 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 1 -- 172.21.15.33:0/3133316942 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f374c05e9a0 con 0x5609853b8810 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 10 client.5724 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 10 client.5724 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T01:09:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 20 client.5724 populate_metadata read hostname 'smithi033' 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x56098542e5f0 0x5609853cb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 4 client.5724 mds_command: new command op to 5520 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 --> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] -- command(tid 0: {"prefix": "status"}) -- 0x5609851c9b20 con 0x56098542e5f0 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f3758cb0640 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x56098542e5f0 0x5609853cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f3758cb0640 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x56098542e5f0 0x5609853cb040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ebb98a2c157a46c server_cookie=bf06f38b46a599f6 in_seq=0 out_seq=0 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 10 client.5724 ms_handle_connect on v2:172.21.15.33:6832/3726672082 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 1 -- 172.21.15.33:0/3133316942 <== mds.0 v2:172.21.15.33:6832/3726672082 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5609851c9b20 con 0x56098542e5f0 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 10 client.5724 handle_command_reply: tid=0 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 client.5724 shutdown 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b8810 msgr2=0x5609853ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b8810 0x5609853ca2d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f374c0012f0 tx=0x7f374c056a10 comp rx=0 tx=0).stop 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 shutdown_connections 2024-04-25T01:09:51.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:6832/3726672082,v1:172.21.15.33:6834/3726672082] conn(0x56098542e5f0 0x5609853cb040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5609853b8810 0x5609853ca2d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x5609853b7ef0 0x560985427cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 --2- 172.21.15.33:0/3133316942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56098536ebc0 0x560985427790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f37337fe640 0 client.5724 ms_handle_reset on v2:172.21.15.33:6832/3726672082 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 >> 172.21.15.33:0/3133316942 conn(0x56098536ab70 msgr2=0x5609853b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 shutdown_connections 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 1 -- 172.21.15.33:0/3133316942 wait complete. 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375e08d000 20 client.5724 trim_cache size 0 max 16384 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:09:51.715 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "10df718f-e6ff-4eb1-9854-6e8a751898ff", 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5520, 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 5.7822734249999996, 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 17, 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 39, 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 39, 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 12.861326857 2024-04-25T01:09:51.716 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:09:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f37400678f0 msgr2=0x7f3740069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f37400678f0 0x7f3740069db0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f37440ad000 tx=0x7f37440a6000 comp rx=0 tx=0).stop 2024-04-25T01:09:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415dcd0 msgr2=0x7f3754072350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415dcd0 0x7f3754072350 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f374c000f90 tx=0x7f374c00b4a0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 shutdown_connections 2024-04-25T01:09:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f37400678f0 0x7f3740069db0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f375415dcd0 0x7f3754072350 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f375415d870 0x7f3754071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 --2- 172.21.15.33:0/1976824246 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f373c005bd0 0x7f375414c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 >> 172.21.15.33:0/1976824246 conn(0x7f37540b29c0 msgr2=0x7f37540b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 shutdown_connections 2024-04-25T01:09:51.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.716+0000 7f375af14640 1 -- 172.21.15.33:0/1976824246 wait complete. 2024-04-25T01:09:51.726 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T01:09:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9db5d82640 1 Processor -- start 2024-04-25T01:09:51.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9db5d82640 1 -- start start 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9db5d82640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9db00b29a0 0x7f9db00b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9db5d82640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db006d9c0 0x7f9db01625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9db5d82640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9db0162c20 con 0x7f9db00ce710 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9db5d82640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f9db00b2ef0 con 0x7f9db00b29a0 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9db5d82640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9db00b3140 con 0x7f9db006d9c0 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daeffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9db00b29a0 0x7f9db00b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daeffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f9db00b29a0 0x7f9db00b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43132/0 (socket says 172.21.15.33:43132) 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daeffd640 1 -- 172.21.15.33:0/1271645601 learned_addr learned my addr 172.21.15.33:0/1271645601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:51.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daffff640 1 --2- 172.21.15.33:0/1271645601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db006d9c0 0x7f9db01625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daeffd640 1 -- 172.21.15.33:0/1271645601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db006d9c0 msgr2=0x7f9db01625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:09:51.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daeffd640 1 --2- 172.21.15.33:0/1271645601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db006d9c0 0x7f9db01625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daeffd640 1 -- 172.21.15.33:0/1271645601 >> v1:172.21.15.138:6789/0 conn(0x7f9db00ce710 legacy=0x7f9db0161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.848+0000 7f9daeffd640 1 -- 172.21.15.33:0/1271645601 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db00b3420 con 0x7f9db00b29a0 2024-04-25T01:09:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9daeffd640 1 --2- 172.21.15.33:0/1271645601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00b29a0 0x7f9db00b4220 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009870 tx=0x7f9da401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb13191f7963c266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9dae7fc640 1 -- 172.21.15.33:0/1271645601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40b0070 con 0x7f9db00b29a0 2024-04-25T01:09:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9dae7fc640 1 -- 172.21.15.33:0/1271645601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da40ab600 con 0x7f9db00b29a0 2024-04-25T01:09:51.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9dae7fc640 1 -- 172.21.15.33:0/1271645601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da40ab9d0 con 0x7f9db00b29a0 2024-04-25T01:09:51.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- 172.21.15.33:0/1271645601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00b29a0 msgr2=0x7f9db00b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1271645601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00b29a0 0x7f9db00b4220 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009870 tx=0x7f9da401c410 comp rx=0 tx=0).stop 2024-04-25T01:09:51.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- 172.21.15.33:0/1271645601 shutdown_connections 2024-04-25T01:09:51.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1271645601 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db006d9c0 0x7f9db01625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1271645601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00b29a0 0x7f9db00b4220 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- 172.21.15.33:0/1271645601 >> 172.21.15.33:0/1271645601 conn(0x7f9db00b3700 msgr2=0x7f9db00b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- 172.21.15.33:0/1271645601 shutdown_connections 2024-04-25T01:09:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- 172.21.15.33:0/1271645601 wait complete. 2024-04-25T01:09:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 Processor -- start 2024-04-25T01:09:51.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- start start 2024-04-25T01:09:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db006d9c0 0x7f9db0156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db00b29a0 0x7f9db00caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00ce710 0x7f9db006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00cce20 con 0x7f9db00ce710 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9db00ccca0 con 0x7f9db006d9c0 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9db5d82640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9db00ccfa0 con 0x7f9db00b29a0 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9daffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00ce710 0x7f9db006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9daffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00ce710 0x7f9db006f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43140/0 (socket says 172.21.15.33:43140) 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.852+0000 7f9daffff640 1 -- 172.21.15.33:0/1706574557 learned_addr learned my addr 172.21.15.33:0/1706574557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daeffd640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db00b29a0 0x7f9db00caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daf7fe640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db006d9c0 0x7f9db0156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daffff640 1 -- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db00b29a0 msgr2=0x7f9db00caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daffff640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db00b29a0 0x7f9db00caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daffff640 1 -- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db006d9c0 msgr2=0x7f9db0156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daffff640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db006d9c0 0x7f9db0156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daffff640 1 -- 172.21.15.33:0/1706574557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db01fadc0 con 0x7f9db00ce710 2024-04-25T01:09:51.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daffff640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00ce710 0x7f9db006f080 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e820 tx=0x7f9da800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9d97fff640 1 -- 172.21.15.33:0/1706574557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8017070 con 0x7f9db00ce710 2024-04-25T01:09:51.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9d97fff640 1 -- 172.21.15.33:0/1706574557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9da800c940 con 0x7f9db00ce710 2024-04-25T01:09:51.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9d97fff640 1 -- 172.21.15.33:0/1706574557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da800cd10 con 0x7f9db00ce710 2024-04-25T01:09:51.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db01fb060 con 0x7f9db00ce710 2024-04-25T01:09:51.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db01fbb40 con 0x7f9db00ce710 2024-04-25T01:09:51.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9db0069040 con 0x7f9db00ce710 2024-04-25T01:09:51.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9d97fff640 1 -- 172.21.15.33:0/1706574557 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9da801a070 con 0x7f9db00ce710 2024-04-25T01:09:51.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9d97fff640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9d9c0678f0 0x7f9d9c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:09:51.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9d97fff640 1 -- 172.21.15.33:0/1706574557 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9da8085cd0 con 0x7f9db00ce710 2024-04-25T01:09:51.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daf7fe640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9d9c0678f0 0x7f9d9c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:09:51.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.856+0000 7f9daf7fe640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9d9c0678f0 0x7f9d9c069db0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001cb0 tx=0x7f9da0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:09:51.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:51.860+0000 7f9d97fff640 1 -- 172.21.15.33:0/1706574557 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9da80565e0 con 0x7f9db00ce710 2024-04-25T01:09:52.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.052+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9db00cbcb0 con 0x7f9db00ce710 2024-04-25T01:09:52.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.052+0000 7f9d97fff640 1 -- 172.21.15.33:0/1706574557 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f9da8055ee0 con 0x7f9db00ce710 2024-04-25T01:09:52.046 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T01:09:52.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9d9c0678f0 msgr2=0x7f9d9c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9d9c0678f0 0x7f9d9c069db0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001cb0 tx=0x7f9da0009040 comp rx=0 tx=0).stop 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00ce710 msgr2=0x7f9db006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00ce710 0x7f9db006f080 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e820 tx=0x7f9da800ece0 comp rx=0 tx=0).stop 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 shutdown_connections 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9d9c0678f0 0x7f9d9c069db0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9db00ce710 0x7f9db006f080 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9db00b29a0 0x7f9db00caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 --2- 172.21.15.33:0/1706574557 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9db006d9c0 0x7f9db0156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 >> 172.21.15.33:0/1706574557 conn(0x7f9db00b3700 msgr2=0x7f9db015e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 shutdown_connections 2024-04-25T01:09:52.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52.056+0000 7f9db5d82640 1 -- 172.21.15.33:0/1706574557 wait complete. 2024-04-25T01:09:52.064 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:09:52.064 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:09:52.064 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-25T01:09:52.064 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T01:09:52.065 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:52.065 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:09:52.065 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:09:52.065 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:09:52.065 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:09:52.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:52.065 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-25T01:09:52.071 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:09:52.071 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:09:52.071 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-25T01:09:52.072 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42411sec preferred_lft 42411sec 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 02:42:26:2b:e6:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.073 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:09:52.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:52.075 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:09:52.129 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-25T01:09:52.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:52.130 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-25T01:09:52.180 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-25T01:09:52.181 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:09:52.181 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T01:09:52.181 INFO:teuthology.orchestra.run.smithi033.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T01:09:52.181 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T01:09:52.181 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:09:52.181 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:09:52.181 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:09:52.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T01:09:52.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T01:09:52.182 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T01:09:52.182 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:09:52.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:09:52.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:52.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:52.303 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:09:52.308 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T01:09:52.309 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T01:09:52.309 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:09:52.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:52.309 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-25T01:09:52.364 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T01:09:52.364 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:09:52.364 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:52.415 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:09:52.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:09:52.416 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:52.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:09:52.521 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:09:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:09:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:09:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:09:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:09:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:09:52.570 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:52.571 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:09:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:09:52.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:52.573 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:09:52.619 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:09:52.620 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T01:09:52.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:09:52.681 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:52.727 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:09:52.727 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:09:52.727 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:09:52.728 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:09:52.729 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:09:52.730 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:09:52.730 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:09:52.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:52.731 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:09:52.749 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2024-04-25T01:09:52.756+0000 7faa58579580 -1 init, newargv = 0x55b644e879c0 newargc=15 2024-04-25T01:09:52.749 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[38878]: starting ceph client 2024-04-25T01:09:52.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[38878]: starting fuse 2024-04-25T01:09:53.855 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:09:53.872 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/run/lock 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:09:53.922 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/fs/nfsd 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/run/rpc_pipefs 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:09:53.923 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:09:53.924 INFO:teuthology.orchestra.run.smithi033.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:53.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:53.925 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:09:53.971 INFO:teuthology.orchestra.run.smithi033.stdout:45 2024-04-25T01:09:53.972 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-25T01:09:53.972 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-25T01:09:53.973 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-25T01:09:53.974 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:09:53.974 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:09:53.974 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2024-04-25T01:09:53.974 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-25T01:09:54.099 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.38878.asok 2024-04-25T01:09:54.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:09:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:09:54.111 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38878.asok 2024-04-25T01:09:54.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:54.111 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38878.asok status 2024-04-25T01:09:54.273 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:09:54.273 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-25T01:09:54.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "d2c45d977142e024a59b68a8f1dce8f0b9840551", 2024-04-25T01:09:54.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T01:09:54.273 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "38878", 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 5736, 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 5736 2024-04-25T01:09:54.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1700713339 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1700713339 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.5736 192.168.0.1:0/1700713339", 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/1700713339", 2024-04-25T01:09:54.275 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-04-25T01:09:54.276 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 17, 2024-04-25T01:09:54.276 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 39, 2024-04-25T01:09:54.276 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:09:54.276 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-25T01:09:54.276 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-25T01:09:54.276 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:09:54.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:54.287 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-25T01:09:54.287 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:54.293 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-25T01:09:54.293 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:54.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:54.294 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:09:54.346 DEBUG:teuthology.orchestra.run.smithi033:> mktemp -d --suffix=xfstests-dev 2024-04-25T01:09:54.396 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.mSBFkuSi7xxfstests-dev 2024-04-25T01:09:54.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:54.396 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.mSBFkuSi7xxfstests-dev) 2024-04-25T01:09:54.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:54.455 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.mSBFkuSi7xxfstests-dev) 2024-04-25T01:09:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:Cloning into '/tmp/tmp.mSBFkuSi7xxfstests-dev'... 2024-04-25T01:09:55.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:55.332 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-25T01:09:55.388 DEBUG:teuthology.orchestra.run.smithi033:> mktemp -d --suffix=test 2024-04-25T01:09:55.438 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.TBpI6gZ746test 2024-04-25T01:09:55.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:09:55.439 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-25T01:09:55.497 DEBUG:teuthology.orchestra.run.smithi033:> mktemp -d --suffix=scratch 2024-04-25T01:09:55.543 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.XxnvMbC4niscratch 2024-04-25T01:09:55.544 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "ubuntu" and "22.04". 2024-04-25T01:09:55.544 DEBUG:teuthology.orchestra.run.smithi033:> sudo apt-get install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc git indent libacl1-dev libaio-dev libcap-dev libgdbm-dev libtool libtool-bin liburing-dev libuuid1 lvm2 make psmisc python3 quota sed uuid-dev uuid-runtime xfsprogs sqlite3 exfatprogs f2fs-tools ocfs2-tools udftools xfsdump xfslibs-dev 2024-04-25T01:09:55.655 INFO:teuthology.orchestra.run.smithi033.stdout:Reading package lists... 2024-04-25T01:09:55.865 INFO:teuthology.orchestra.run.smithi033.stdout:Building dependency tree... 2024-04-25T01:09:55.866 INFO:teuthology.orchestra.run.smithi033.stdout:Reading state information... 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:acl is already the newest version (2.3.1-1). 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:acl set to manually installed. 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:attr is already the newest version (1:2.5.1-1build1). 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:automake is already the newest version (1:1.16.5-1.3). 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:bc is already the newest version (1.07.1-3build1). 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:gcc is already the newest version (4:11.2.0-1ubuntu1). 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:gcc set to manually installed. 2024-04-25T01:09:56.074 INFO:teuthology.orchestra.run.smithi033.stdout:libacl1-dev is already the newest version (2.3.1-1). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:libaio-dev is already the newest version (0.3.112-13build1). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:libtool is already the newest version (2.4.6-15build2). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:lvm2 is already the newest version (2.03.11-2.1ubuntu4). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:make is already the newest version (4.3-4.1build1). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:make set to manually installed. 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:psmisc is already the newest version (23.4-2build3). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:quota is already the newest version (4.06-1build2). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:sed is already the newest version (4.8-1ubuntu2). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:xfsdump is already the newest version (3.1.9+0+nmu2). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:xfsprogs is already the newest version (5.13.0-1ubuntu2). 2024-04-25T01:09:56.075 INFO:teuthology.orchestra.run.smithi033.stdout:dbench is already the newest version (4.0-2build1). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:dump is already the newest version (0.4b47-3). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:e2fsprogs is already the newest version (1.46.5-2ubuntu1.1). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:git is already the newest version (1:2.34.1-1ubuntu1.10). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:indent is already the newest version (2.2.12-1ubuntu0.22.04.1). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:libuuid1 is already the newest version (2.37.2-4ubuntu3.4). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:python3 is already the newest version (3.10.6-1~22.04). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3.4). 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libiscsi7 libpmemobj1 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: libpython2-dev libpython2-stdlib libpython2.7 libpython2.7-dev 2024-04-25T01:09:56.076 INFO:teuthology.orchestra.run.smithi033.stdout: libpython2.7-minimal libpython2.7-stdlib linux-image-5.4.0-135-generic 2024-04-25T01:09:56.077 INFO:teuthology.orchestra.run.smithi033.stdout: linux-modules-5.4.0-135-generic python2 python2-dev python2-minimal 2024-04-25T01:09:56.077 INFO:teuthology.orchestra.run.smithi033.stdout: python2.7 python2.7-dev python2.7-minimal 2024-04-25T01:09:56.077 INFO:teuthology.orchestra.run.smithi033.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T01:09:56.077 INFO:teuthology.orchestra.run.smithi033.stdout:The following additional packages will be installed: 2024-04-25T01:09:56.078 INFO:teuthology.orchestra.run.smithi033.stdout: libcap2 libcmap4 libcorosync-common4 libdlm3 libpmemblk1 libqb100 2024-04-25T01:09:56.078 INFO:teuthology.orchestra.run.smithi033.stdout: libsqlite3-0 2024-04-25T01:09:56.078 INFO:teuthology.orchestra.run.smithi033.stdout:Suggested packages: 2024-04-25T01:09:56.078 INFO:teuthology.orchestra.run.smithi033.stdout: gnuplot gfio python-scipy gawk-doc sqlite3-doc dvd+rw-tools udfclient 2024-04-25T01:09:56.113 INFO:teuthology.orchestra.run.smithi033.stdout:The following NEW packages will be installed: 2024-04-25T01:09:56.113 INFO:teuthology.orchestra.run.smithi033.stdout: exfatprogs f2fs-tools fio libcap-dev libcmap4 libcorosync-common4 libdlm3 2024-04-25T01:09:56.114 INFO:teuthology.orchestra.run.smithi033.stdout: libgdbm-dev libpmemblk1 libqb100 liburing-dev ocfs2-tools sqlite3 udftools 2024-04-25T01:09:56.114 INFO:teuthology.orchestra.run.smithi033.stdout:The following packages will be upgraded: 2024-04-25T01:09:56.116 INFO:teuthology.orchestra.run.smithi033.stdout: gawk libcap2 libsqlite3-0 uuid-runtime 2024-04-25T01:09:56.322 INFO:teuthology.orchestra.run.smithi033.stdout:4 upgraded, 14 newly installed, 0 to remove and 294 not upgraded. 2024-04-25T01:09:56.322 INFO:teuthology.orchestra.run.smithi033.stdout:Need to get 7,314 kB of archives. 2024-04-25T01:09:56.322 INFO:teuthology.orchestra.run.smithi033.stdout:After this operation, 14.2 MB of additional disk space will be used. 2024-04-25T01:09:56.322 INFO:teuthology.orchestra.run.smithi033.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-runtime amd64 2.37.2-4ubuntu3.4 [32.0 kB] 2024-04-25T01:09:56.526 INFO:teuthology.orchestra.run.smithi033.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 gawk amd64 1:5.1.0-1ubuntu0.1 [447 kB] 2024-04-25T01:09:56.864 INFO:teuthology.orchestra.run.smithi033.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcorosync-common4 amd64 3.1.6-1ubuntu1 [7,322 B] 2024-04-25T01:09:56.865 INFO:teuthology.orchestra.run.smithi033.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqb100 amd64 2.0.4-1ubuntu0.2 [74.2 kB] 2024-04-25T01:09:56.883 INFO:teuthology.orchestra.run.smithi033.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 libcmap4 amd64 3.1.6-1ubuntu1 [10.4 kB] 2024-04-25T01:09:56.885 INFO:teuthology.orchestra.run.smithi033.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 libdlm3 amd64 4.1.1-1build1 [14.4 kB] 2024-04-25T01:09:56.888 INFO:teuthology.orchestra.run.smithi033.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 ocfs2-tools amd64 1.8.7-1build1 [646 kB] 2024-04-25T01:09:56.979 INFO:teuthology.orchestra.run.smithi033.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap2 amd64 1:2.44-1ubuntu0.22.04.1 [18.3 kB] 2024-04-25T01:09:56.980 INFO:teuthology.orchestra.run.smithi033.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libsqlite3-0 amd64 3.37.2-2ubuntu0.3 [641 kB] 2024-04-25T01:09:57.034 INFO:teuthology.orchestra.run.smithi033.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 exfatprogs amd64 1.1.3-1 [40.3 kB] 2024-04-25T01:09:57.034 INFO:teuthology.orchestra.run.smithi033.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 f2fs-tools amd64 1.14.0-2build1 [169 kB] 2024-04-25T01:09:57.208 INFO:teuthology.orchestra.run.smithi033.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libpmemblk1 amd64 1.11.1-3build1 [65.6 kB] 2024-04-25T01:09:57.495 INFO:teuthology.orchestra.run.smithi033.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 fio amd64 3.28-1 [4,047 kB] 2024-04-25T01:09:58.004 INFO:teuthology.orchestra.run.smithi033.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcap-dev amd64 1:2.44-1ubuntu0.22.04.1 [39.4 kB] 2024-04-25T01:09:58.005 INFO:teuthology.orchestra.run.smithi033.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libgdbm-dev amd64 1.23-1 [117 kB] 2024-04-25T01:09:58.008 INFO:teuthology.orchestra.run.smithi033.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liburing-dev amd64 2.1-2build1 [51.8 kB] 2024-04-25T01:09:58.010 INFO:teuthology.orchestra.run.smithi033.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 sqlite3 amd64 3.37.2-2ubuntu0.3 [768 kB] 2024-04-25T01:09:58.028 INFO:teuthology.orchestra.run.smithi033.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 udftools amd64 2.3-1 [125 kB] 2024-04-25T01:09:58.338 INFO:teuthology.orchestra.run.smithi033.stderr:debconf: unable to initialize frontend: Dialog 2024-04-25T01:09:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-04-25T01:09:58.339 INFO:teuthology.orchestra.run.smithi033.stderr:debconf: falling back to frontend: Readline 2024-04-25T01:09:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:debconf: unable to initialize frontend: Readline 2024-04-25T01:09:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:debconf: (This frontend requires a controlling tty.) 2024-04-25T01:09:58.356 INFO:teuthology.orchestra.run.smithi033.stderr:debconf: falling back to frontend: Teletype 2024-04-25T01:09:58.360 INFO:teuthology.orchestra.run.smithi033.stderr:dpkg-preconfigure: unable to re-open stdin: 2024-04-25T01:09:58.396 INFO:teuthology.orchestra.run.smithi033.stdout:Fetched 7,314 kB in 2s (3,836 kB/s) 2024-04-25T01:09:58.723 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176786 files and directories currently installed.) 2024-04-25T01:09:58.729 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../0-uuid-runtime_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T01:09:58.861 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking uuid-runtime (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T01:09:59.314 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../1-gawk_1%3a5.1.0-1ubuntu0.1_amd64.deb ... 2024-04-25T01:09:59.451 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking gawk (1:5.1.0-1ubuntu0.1) over (1:5.1.0-1build3) ... 2024-04-25T01:10:00.020 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcorosync-common4:amd64. 2024-04-25T01:10:00.050 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../2-libcorosync-common4_3.1.6-1ubuntu1_amd64.deb ... 2024-04-25T01:10:00.112 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T01:10:00.251 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:00.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:00.466 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libqb100:amd64. 2024-04-25T01:10:00.496 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../3-libqb100_2.0.4-1ubuntu0.2_amd64.deb ... 2024-04-25T01:10:00.541 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-04-25T01:10:00.886 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcmap4:amd64. 2024-04-25T01:10:00.915 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../4-libcmap4_3.1.6-1ubuntu1_amd64.deb ... 2024-04-25T01:10:00.953 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T01:10:01.223 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libdlm3:amd64. 2024-04-25T01:10:01.252 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../5-libdlm3_4.1.1-1build1_amd64.deb ... 2024-04-25T01:10:01.289 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libdlm3:amd64 (4.1.1-1build1) ... 2024-04-25T01:10:01.693 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package ocfs2-tools. 2024-04-25T01:10:01.722 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../6-ocfs2-tools_1.8.7-1build1_amd64.deb ... 2024-04-25T01:10:01.759 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking ocfs2-tools (1.8.7-1build1) ... 2024-04-25T01:10:02.177 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../7-libcap2_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:10:02.341 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) over (1:2.44-1build3) ... 2024-04-25T01:10:02.845 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcap2:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-25T01:10:03.342 INFO:teuthology.orchestra.run.smithi033.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 176860 files and directories currently installed.) 2024-04-25T01:10:03.349 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../0-libsqlite3-0_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T01:10:03.516 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) over (3.37.2-2ubuntu0.1) ... 2024-04-25T01:10:03.895 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package exfatprogs. 2024-04-25T01:10:03.924 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../1-exfatprogs_1.1.3-1_amd64.deb ... 2024-04-25T01:10:03.961 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking exfatprogs (1.1.3-1) ... 2024-04-25T01:10:04.240 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package f2fs-tools. 2024-04-25T01:10:04.269 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../2-f2fs-tools_1.14.0-2build1_amd64.deb ... 2024-04-25T01:10:04.365 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking f2fs-tools (1.14.0-2build1) ... 2024-04-25T01:10:04.686 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libpmemblk1:amd64. 2024-04-25T01:10:04.714 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../3-libpmemblk1_1.11.1-3build1_amd64.deb ... 2024-04-25T01:10:04.744 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libpmemblk1:amd64 (1.11.1-3build1) ... 2024-04-25T01:10:05.131 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package fio. 2024-04-25T01:10:05.160 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../4-fio_3.28-1_amd64.deb ... 2024-04-25T01:10:05.197 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking fio (3.28-1) ... 2024-04-25T01:10:05.627 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libcap-dev:amd64. 2024-04-25T01:10:05.653 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../5-libcap-dev_1%3a2.44-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T01:10:05.693 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-25T01:10:05.940 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package libgdbm-dev:amd64. 2024-04-25T01:10:05.957 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../6-libgdbm-dev_1.23-1_amd64.deb ... 2024-04-25T01:10:05.990 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking libgdbm-dev:amd64 (1.23-1) ... 2024-04-25T01:10:06.327 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package liburing-dev:amd64. 2024-04-25T01:10:06.344 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../7-liburing-dev_2.1-2build1_amd64.deb ... 2024-04-25T01:10:06.385 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking liburing-dev:amd64 (2.1-2build1) ... 2024-04-25T01:10:06.689 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package sqlite3. 2024-04-25T01:10:06.718 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../8-sqlite3_3.37.2-2ubuntu0.3_amd64.deb ... 2024-04-25T01:10:06.763 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T01:10:07.134 INFO:teuthology.orchestra.run.smithi033.stdout:Selecting previously unselected package udftools. 2024-04-25T01:10:07.163 INFO:teuthology.orchestra.run.smithi033.stdout:Preparing to unpack .../9-udftools_2.3-1_amd64.deb ... 2024-04-25T01:10:07.213 INFO:teuthology.orchestra.run.smithi033.stdout:Unpacking udftools (2.3-1) ... 2024-04-25T01:10:07.530 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up exfatprogs (1.1.3-1) ... 2024-04-25T01:10:07.655 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up gawk (1:5.1.0-1ubuntu0.1) ... 2024-04-25T01:10:07.891 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up f2fs-tools (1.14.0-2build1) ... 2024-04-25T01:10:08.001 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libsqlite3-0:amd64 (3.37.2-2ubuntu0.3) ... 2024-04-25T01:10:08.127 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libqb100:amd64 (2.0.4-1ubuntu0.2) ... 2024-04-25T01:10:08.253 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcorosync-common4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T01:10:08.379 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libpmemblk1:amd64 (1.11.1-3build1) ... 2024-04-25T01:10:08.505 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up uuid-runtime (2.37.2-4ubuntu3.4) ... 2024-04-25T01:10:09.385 INFO:teuthology.orchestra.run.smithi033.stdout:uuidd.service is a disabled or a static unit not running, not starting it. 2024-04-25T01:10:09.441 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up udftools (2.3-1) ... 2024-04-25T01:10:09.583 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libdlm3:amd64 (4.1.1-1build1) ... 2024-04-25T01:10:09.684 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libgdbm-dev:amd64 (1.23-1) ... 2024-04-25T01:10:09.786 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcap-dev:amd64 (1:2.44-1ubuntu0.22.04.1) ... 2024-04-25T01:10:09.911 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up sqlite3 (3.37.2-2ubuntu0.3) ... 2024-04-25T01:10:10.063 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up liburing-dev:amd64 (2.1-2build1) ... 2024-04-25T01:10:10.189 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up libcmap4:amd64 (3.1.6-1ubuntu1) ... 2024-04-25T01:10:10.306 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up fio (3.28-1) ... 2024-04-25T01:10:10.783 INFO:teuthology.orchestra.run.smithi033.stdout:Setting up ocfs2-tools (1.8.7-1build1) ... 2024-04-25T01:10:11.030 INFO:teuthology.orchestra.run.smithi033.stdout:debconf: unable to initialize frontend: Dialog 2024-04-25T01:10:11.030 INFO:teuthology.orchestra.run.smithi033.stdout:debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) 2024-04-25T01:10:11.030 INFO:teuthology.orchestra.run.smithi033.stdout:debconf: falling back to frontend: Readline 2024-04-25T01:10:11.812 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for initramfs-tools (0.140ubuntu13.1) ... 2024-04-25T01:10:11.928 INFO:teuthology.orchestra.run.smithi033.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T01:10:30.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:30.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:10:30.842 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T01:10:30.994 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T01:10:32.238 INFO:teuthology.orchestra.run.smithi033.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T01:10:32.723 DEBUG:teuthology.orchestra.run.smithi033:> sudo useradd -m fsgqa 2024-04-25T01:10:32.738 INFO:teuthology.orchestra.run.smithi033.stderr:useradd: user 'fsgqa' already exists 2024-04-25T01:10:32.739 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-25T01:10:32.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo groupadd fsgqa 2024-04-25T01:10:32.793 INFO:teuthology.orchestra.run.smithi033.stderr:groupadd: group 'fsgqa' already exists 2024-04-25T01:10:32.795 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-25T01:10:32.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo useradd fsgqa2 2024-04-25T01:10:33.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo useradd 123456-fsgqa 2024-04-25T01:10:33.595 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-25T01:10:33.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfaec2f640 1 Processor -- start 2024-04-25T01:10:33.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfaec2f640 1 -- start start 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfaec2f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdfa815c160 0x7fdfa80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfaec2f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfa80b5360 0x7fdfa80cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfaec2f640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfa815f9a0 con 0x7fdfa8145470 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfaec2f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa80b57b0 con 0x7fdfa80b5360 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfaec2f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdfa815f750 con 0x7fdfa815c160 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfad1cc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfa80b5360 0x7fdfa80cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfad1cc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdfa80b5360 0x7fdfa80cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41440/0 (socket says 172.21.15.33:41440) 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfad1cc640 1 -- 172.21.15.33:0/3116133601 learned_addr learned my addr 172.21.15.33:0/3116133601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:10:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfa7fff640 1 --2- 172.21.15.33:0/3116133601 >> v2:172.21.15.138:3300/0 conn(0x7fdfa815c160 0x7fdfa80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfad1cc640 1 -- 172.21.15.33:0/3116133601 >> v2:172.21.15.138:3300/0 conn(0x7fdfa815c160 msgr2=0x7fdfa80b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:10:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfad1cc640 1 --2- 172.21.15.33:0/3116133601 >> v2:172.21.15.138:3300/0 conn(0x7fdfa815c160 0x7fdfa80b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfad1cc640 1 -- 172.21.15.33:0/3116133601 >> v1:172.21.15.138:6789/0 conn(0x7fdfa8145470 legacy=0x7fdfa80cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:10:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.731+0000 7fdfad1cc640 1 -- 172.21.15.33:0/3116133601 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa815fc80 con 0x7fdfa80b5360 2024-04-25T01:10:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfad1cc640 1 --2- 172.21.15.33:0/3116133601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa80b5360 0x7fdfa80cce00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80cc3c0 tx=0x7fdfa001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a49134515fc14868 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfa77fe640 1 -- 172.21.15.33:0/3116133601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00ab5f0 con 0x7fdfa80b5360 2024-04-25T01:10:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfa77fe640 1 -- 172.21.15.33:0/3116133601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa00b0070 con 0x7fdfa80b5360 2024-04-25T01:10:33.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfa77fe640 1 -- 172.21.15.33:0/3116133601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00ab9b0 con 0x7fdfa80b5360 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- 172.21.15.33:0/3116133601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa80b5360 msgr2=0x7fdfa80cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/3116133601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa80b5360 0x7fdfa80cce00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80cc3c0 tx=0x7fdfa001c4e0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- 172.21.15.33:0/3116133601 shutdown_connections 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/3116133601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa80b5360 0x7fdfa80cce00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/3116133601 >> v2:172.21.15.138:3300/0 conn(0x7fdfa815c160 0x7fdfa80b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- 172.21.15.33:0/3116133601 >> 172.21.15.33:0/3116133601 conn(0x7fdfa80b1810 msgr2=0x7fdfa80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- 172.21.15.33:0/3116133601 shutdown_connections 2024-04-25T01:10:33.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- 172.21.15.33:0/3116133601 wait complete. 2024-04-25T01:10:33.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 Processor -- start 2024-04-25T01:10:33.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- start start 2024-04-25T01:10:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfa80b5360 0x7fdfa813b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa8145470 0x7fdfa80b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa815c160 0x7fdfa815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa806e850 con 0x7fdfa8145470 2024-04-25T01:10:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa8145850 con 0x7fdfa815c160 2024-04-25T01:10:33.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfaec2f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa8168af0 con 0x7fdfa80b5360 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa8145470 0x7fdfa80b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa8145470 0x7fdfa80b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41444/0 (socket says 172.21.15.33:41444) 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfa7fff640 1 -- 172.21.15.33:0/65113622 learned_addr learned my addr 172.21.15.33:0/65113622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfac9cb640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfa80b5360 0x7fdfa813b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.735+0000 7fdfad1cc640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa815c160 0x7fdfa815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa7fff640 1 -- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfa80b5360 msgr2=0x7fdfa813b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa7fff640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfa80b5360 0x7fdfa813b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa7fff640 1 -- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa815c160 msgr2=0x7fdfa815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa7fff640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa815c160 0x7fdfa815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa7fff640 1 -- 172.21.15.33:0/65113622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa813bd70 con 0x7fdfa8145470 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa7fff640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa8145470 0x7fdfa80b3e00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdf98007ec0 tx=0x7fdf9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa57fa640 1 -- 172.21.15.33:0/65113622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98014620 con 0x7fdfa8145470 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa57fa640 1 -- 172.21.15.33:0/65113622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf98019070 con 0x7fdfa8145470 2024-04-25T01:10:33.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa80b46e0 con 0x7fdfa8145470 2024-04-25T01:10:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa57fa640 1 -- 172.21.15.33:0/65113622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98014a20 con 0x7fdfa8145470 2024-04-25T01:10:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa813c010 con 0x7fdfa8145470 2024-04-25T01:10:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfa8069040 con 0x7fdfa8145470 2024-04-25T01:10:33.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa57fa640 1 -- 172.21.15.33:0/65113622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf98014bc0 con 0x7fdfa8145470 2024-04-25T01:10:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa57fa640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdf880678c0 0x7fdf88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfa57fa640 1 -- 172.21.15.33:0/65113622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fdf980866f0 con 0x7fdfa8145470 2024-04-25T01:10:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfac9cb640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdf880678c0 0x7fdf88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:33.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.739+0000 7fdfac9cb640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdf880678c0 0x7fdf88069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf90004e30 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:33.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.743+0000 7fdfa57fa640 1 -- 172.21.15.33:0/65113622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdf98057110 con 0x7fdfa8145470 2024-04-25T01:10:33.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.979+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdfa80b4c00 con 0x7fdfa8145470 2024-04-25T01:10:33.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.979+0000 7fdfa57fa640 1 -- 172.21.15.33:0/65113622 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1285 (secure 0 0 0) 0x7fdf98056a10 con 0x7fdfa8145470 2024-04-25T01:10:33.972 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:10:33.972 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","modified":"2024-04-25T01:07:31.559252Z","created":"2024-04-25T01:07:31.559252Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3301","nonce":0},{"type":"v1","addr":"172.21.15.138:6790","nonce":0}]},"addr":"172.21.15.138:6790/0","public_addr":"172.21.15.138:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-25T01:10:33.972 INFO:teuthology.orchestra.run.smithi033.stderr:dumped monmap epoch 1 2024-04-25T01:10:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdf880678c0 msgr2=0x7fdf88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:33.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdf880678c0 0x7fdf88069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf90004e30 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa8145470 msgr2=0x7fdfa80b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa8145470 0x7fdfa80b3e00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdf98007ec0 tx=0x7fdf9800e4b0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 shutdown_connections 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdf880678c0 0x7fdf88069d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa815c160 0x7fdfa815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa8145470 0x7fdfa80b3e00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 --2- 172.21.15.33:0/65113622 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdfa80b5360 0x7fdfa813b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 >> 172.21.15.33:0/65113622 conn(0x7fdfa80b1810 msgr2=0x7fdfa80cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 shutdown_connections 2024-04-25T01:10:33.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:33.983+0000 7fdfaec2f640 1 -- 172.21.15.33:0/65113622 wait complete. 2024-04-25T01:10:33.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-25T01:10:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.123+0000 7f18719da640 1 Processor -- start 2024-04-25T01:10:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.123+0000 7f18719da640 1 -- start start 2024-04-25T01:10:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f18719da640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f18719da640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f186c0b5360 0x7f186c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f18719da640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f186c15f9a0 con 0x7f186c145470 2024-04-25T01:10:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f18719da640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f186c0b57b0 con 0x7f186c0b5360 2024-04-25T01:10:34.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f18719da640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f186c15f750 con 0x7f186c15c160 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f186c0b5360 0x7f186c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f186c0b5360 0x7f186c0cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41456/0 (socket says 172.21.15.33:41456) 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 -- 172.21.15.33:0/1120978207 learned_addr learned my addr 172.21.15.33:0/1120978207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186a7fc640 1 --2- 172.21.15.33:0/1120978207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 -- 172.21.15.33:0/1120978207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 msgr2=0x7f186c0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 --2- 172.21.15.33:0/1120978207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c0b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 -- 172.21.15.33:0/1120978207 >> v1:172.21.15.138:6789/0 conn(0x7f186c145470 legacy=0x7f186c0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:34.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 -- 172.21.15.33:0/1120978207 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c15fc80 con 0x7f186c0b5360 2024-04-25T01:10:34.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f186b7fe640 1 --2- 172.21.15.33:0/1120978207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c0cce00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e50 tx=0x7f186401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bee409bfdf0ec95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:34.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f1869ffb640 1 -- 172.21.15.33:0/1120978207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640a5810 con 0x7f186c0b5360 2024-04-25T01:10:34.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f1869ffb640 1 -- 172.21.15.33:0/1120978207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18640a59b0 con 0x7f186c0b5360 2024-04-25T01:10:34.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.127+0000 7f1869ffb640 1 -- 172.21.15.33:0/1120978207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18640a5d80 con 0x7f186c0b5360 2024-04-25T01:10:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- 172.21.15.33:0/1120978207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 msgr2=0x7f186c0cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 --2- 172.21.15.33:0/1120978207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c0cce00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1864009e50 tx=0x7f186401ca30 comp rx=0 tx=0).stop 2024-04-25T01:10:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- 172.21.15.33:0/1120978207 shutdown_connections 2024-04-25T01:10:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 --2- 172.21.15.33:0/1120978207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c0cce00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 --2- 172.21.15.33:0/1120978207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c0b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- 172.21.15.33:0/1120978207 >> 172.21.15.33:0/1120978207 conn(0x7f186c0b1810 msgr2=0x7f186c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:34.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- 172.21.15.33:0/1120978207 shutdown_connections 2024-04-25T01:10:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- 172.21.15.33:0/1120978207 wait complete. 2024-04-25T01:10:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 Processor -- start 2024-04-25T01:10:34.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- start start 2024-04-25T01:10:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c13b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186c145470 0x7f186c15e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f186c06e850 con 0x7f186c0b5360 2024-04-25T01:10:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f186c145850 con 0x7f186c145470 2024-04-25T01:10:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f186c168af0 con 0x7f186c15c160 2024-04-25T01:10:34.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c13b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c13b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41464/0 (socket says 172.21.15.33:41464) 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 -- 172.21.15.33:0/2027944107 learned_addr learned my addr 172.21.15.33:0/2027944107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186b7fe640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 -- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 msgr2=0x7f186c144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 -- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186c145470 msgr2=0x7f186c15e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186a7fc640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186c145470 0x7f186c15e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186c145470 0x7f186c15e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 -- 172.21.15.33:0/2027944107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c13a970 con 0x7f186c0b5360 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186a7fc640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186c145470 0x7f186c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f186affd640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c13b540 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f186000c370 tx=0x7f186000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f1853fff640 1 -- 172.21.15.33:0/2027944107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600052d0 con 0x7f186c0b5360 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f1853fff640 1 -- 172.21.15.33:0/2027944107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1860005470 con 0x7f186c0b5360 2024-04-25T01:10:34.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f1853fff640 1 -- 172.21.15.33:0/2027944107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860007260 con 0x7f186c0b5360 2024-04-25T01:10:34.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c13c3c0 con 0x7f186c0b5360 2024-04-25T01:10:34.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.131+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c13c980 con 0x7f186c0b5360 2024-04-25T01:10:34.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.135+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f186c069040 con 0x7f186c0b5360 2024-04-25T01:10:34.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.135+0000 7f1853fff640 1 -- 172.21.15.33:0/2027944107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1860007400 con 0x7f186c0b5360 2024-04-25T01:10:34.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.135+0000 7f1853fff640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18580678f0 0x7f1858069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:10:34.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.135+0000 7f1853fff640 1 -- 172.21.15.33:0/2027944107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f1860085200 con 0x7f186c0b5360 2024-04-25T01:10:34.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.135+0000 7f186a7fc640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18580678f0 0x7f1858069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:10:34.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.135+0000 7f186a7fc640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18580678f0 0x7f1858069db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f186c144e90 tx=0x7f185c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:10:34.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.139+0000 7f1853fff640 1 -- 172.21.15.33:0/2027944107 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1860055c20 con 0x7f186c0b5360 2024-04-25T01:10:34.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f186c0b15f0 con 0x7f186c0b5360 2024-04-25T01:10:34.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f1853fff640 1 -- 172.21.15.33:0/2027944107 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v13) ==== 82+0+151 (secure 0 0 0) 0x7f1860055520 con 0x7f186c0b5360 2024-04-25T01:10:34.351 INFO:teuthology.orchestra.run.smithi033.stdout:[client.admin] 2024-04-25T01:10:34.351 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDTrClmXiTaLBAAL67Nc3L8d18iT5X9HoT7yA== 2024-04-25T01:10:34.351 INFO:teuthology.orchestra.run.smithi033.stdout: caps mds = "allow *" 2024-04-25T01:10:34.352 INFO:teuthology.orchestra.run.smithi033.stdout: caps mgr = "allow *" 2024-04-25T01:10:34.352 INFO:teuthology.orchestra.run.smithi033.stdout: caps mon = "allow *" 2024-04-25T01:10:34.352 INFO:teuthology.orchestra.run.smithi033.stdout: caps osd = "allow *" 2024-04-25T01:10:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18580678f0 msgr2=0x7f1858069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:34.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18580678f0 0x7f1858069db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f186c144e90 tx=0x7f185c008040 comp rx=0 tx=0).stop 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 msgr2=0x7f186c13b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c13b540 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f186000c370 tx=0x7f186000c830 comp rx=0 tx=0).stop 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 shutdown_connections 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f18580678f0 0x7f1858069db0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f186c15c160 0x7f186c144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186c145470 0x7f186c15e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 --2- 172.21.15.33:0/2027944107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f186c0b5360 0x7f186c13b540 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 >> 172.21.15.33:0/2027944107 conn(0x7f186c0b1810 msgr2=0x7f186c0722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 shutdown_connections 2024-04-25T01:10:34.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:10:34.359+0000 7f18719da640 1 -- 172.21.15.33:0/2027944107 wait complete. 2024-04-25T01:10:34.371 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-04-25T01:10:34.377 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.Wg7fpwcQCN 2024-04-25T01:10:34.377 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:10:34.377 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/tmp/tmp.Wg7fpwcQCN 2024-04-25T01:10:34.426 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:10:34.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/tmp/tmp.mSBFkuSi7xxfstests-dev/local.config 2024-04-25T01:10:34.482 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.TBpI6gZ746test export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.XxnvMbC4niscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.33:6789 -k /tmp/tmp.Wg7fpwcQCN --client_mountpoint /test" 2024-04-25T01:10:34.483 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:10:34.483 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/tmp/tmp.mSBFkuSi7xxfstests-dev/ceph.exclude 2024-04-25T01:10:34.575 DEBUG:teuthology.orchestra.run.smithi033:> (cd /tmp/tmp.mSBFkuSi7xxfstests-dev && exec sudo make) 2024-04-25T01:10:34.640 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize -cfi 2024-04-25T01:10:34.908 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T01:10:34.908 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file './config.guess' 2024-04-25T01:10:34.913 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file './config.sub' 2024-04-25T01:10:34.924 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file './install-sh' 2024-04-25T01:10:34.931 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T01:10:34.953 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T01:10:34.954 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T01:10:34.965 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T01:10:34.971 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T01:10:34.976 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T01:10:34.981 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T01:10:34.986 INFO:teuthology.orchestra.run.smithi033.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T01:10:34.987 INFO:teuthology.orchestra.run.smithi033.stdout:cp include/install-sh . 2024-04-25T01:10:34.988 INFO:teuthology.orchestra.run.smithi033.stdout:aclocal -I m4 2024-04-25T01:10:36.159 INFO:teuthology.orchestra.run.smithi033.stdout:autoheader 2024-04-25T01:10:36.516 INFO:teuthology.orchestra.run.smithi033.stdout:autoconf 2024-04-25T01:10:36.925 INFO:teuthology.orchestra.run.smithi033.stdout:./configure \ 2024-04-25T01:10:36.925 INFO:teuthology.orchestra.run.smithi033.stdout: --libexecdir=/usr/lib \ 2024-04-25T01:10:36.925 INFO:teuthology.orchestra.run.smithi033.stdout: --exec_prefix=/var/lib 2024-04-25T01:10:37.337 INFO:teuthology.orchestra.run.smithi033.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T01:10:37.337 INFO:teuthology.orchestra.run.smithi033.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T01:10:37.382 INFO:teuthology.orchestra.run.smithi033.stdout:checking how to print strings... printf 2024-04-25T01:10:37.382 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gcc... gcc 2024-04-25T01:10:37.624 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether the C compiler works... yes 2024-04-25T01:10:37.624 INFO:teuthology.orchestra.run.smithi033.stdout:checking for C compiler default output file name... a.out 2024-04-25T01:10:37.664 INFO:teuthology.orchestra.run.smithi033.stdout:checking for suffix of executables... 2024-04-25T01:10:37.740 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether we are cross compiling... no 2024-04-25T01:10:37.761 INFO:teuthology.orchestra.run.smithi033.stdout:checking for suffix of object files... o 2024-04-25T01:10:37.778 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether the compiler supports GNU C... yes 2024-04-25T01:10:37.797 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether gcc accepts -g... yes 2024-04-25T01:10:37.933 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gcc option to enable C11 features... none needed 2024-04-25T01:10:37.937 INFO:teuthology.orchestra.run.smithi033.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-25T01:10:37.942 INFO:teuthology.orchestra.run.smithi033.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T01:10:37.944 INFO:teuthology.orchestra.run.smithi033.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T01:10:37.945 INFO:teuthology.orchestra.run.smithi033.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-25T01:10:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-25T01:10:37.954 INFO:teuthology.orchestra.run.smithi033.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-25T01:10:37.964 INFO:teuthology.orchestra.run.smithi033.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-25T01:10:37.988 INFO:teuthology.orchestra.run.smithi033.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-25T01:10:37.989 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether ln -s works... yes 2024-04-25T01:10:38.000 INFO:teuthology.orchestra.run.smithi033.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T01:10:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T01:10:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T01:10:38.004 INFO:teuthology.orchestra.run.smithi033.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-25T01:10:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:checking for objdump... objdump 2024-04-25T01:10:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T01:10:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:checking for dlltool... no 2024-04-25T01:10:38.005 INFO:teuthology.orchestra.run.smithi033.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T01:10:38.006 INFO:teuthology.orchestra.run.smithi033.stdout:checking for ar... ar 2024-04-25T01:10:38.046 INFO:teuthology.orchestra.run.smithi033.stdout:checking for archiver @FILE support... @ 2024-04-25T01:10:38.046 INFO:teuthology.orchestra.run.smithi033.stdout:checking for strip... strip 2024-04-25T01:10:38.046 INFO:teuthology.orchestra.run.smithi033.stdout:checking for ranlib... ranlib 2024-04-25T01:10:38.047 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gawk... gawk 2024-04-25T01:10:38.118 INFO:teuthology.orchestra.run.smithi033.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-25T01:10:38.120 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sysroot... no 2024-04-25T01:10:38.124 INFO:teuthology.orchestra.run.smithi033.stdout:checking for a working dd... /usr/bin/dd 2024-04-25T01:10:38.128 INFO:teuthology.orchestra.run.smithi033.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-25T01:10:38.144 INFO:teuthology.orchestra.run.smithi033.stdout:checking for mt... mt 2024-04-25T01:10:38.158 INFO:teuthology.orchestra.run.smithi033.stdout:checking if mt is a manifest tool... no 2024-04-25T01:10:38.181 INFO:teuthology.orchestra.run.smithi033.stdout:checking for stdio.h... yes 2024-04-25T01:10:38.220 INFO:teuthology.orchestra.run.smithi033.stdout:checking for stdlib.h... yes 2024-04-25T01:10:38.248 INFO:teuthology.orchestra.run.smithi033.stdout:checking for string.h... yes 2024-04-25T01:10:38.285 INFO:teuthology.orchestra.run.smithi033.stdout:checking for inttypes.h... yes 2024-04-25T01:10:38.313 INFO:teuthology.orchestra.run.smithi033.stdout:checking for stdint.h... yes 2024-04-25T01:10:38.338 INFO:teuthology.orchestra.run.smithi033.stdout:checking for strings.h... yes 2024-04-25T01:10:38.369 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/stat.h... yes 2024-04-25T01:10:38.394 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/types.h... yes 2024-04-25T01:10:38.434 INFO:teuthology.orchestra.run.smithi033.stdout:checking for unistd.h... yes 2024-04-25T01:10:38.466 INFO:teuthology.orchestra.run.smithi033.stdout:checking for dlfcn.h... yes 2024-04-25T01:10:38.469 INFO:teuthology.orchestra.run.smithi033.stdout:checking for objdir... .libs 2024-04-25T01:10:38.555 INFO:teuthology.orchestra.run.smithi033.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T01:10:38.555 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T01:10:38.587 INFO:teuthology.orchestra.run.smithi033.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T01:10:38.834 INFO:teuthology.orchestra.run.smithi033.stdout:checking if gcc static flag -static works... yes 2024-04-25T01:10:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T01:10:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T01:10:38.875 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T01:10:38.903 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T01:10:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T01:10:38.988 INFO:teuthology.orchestra.run.smithi033.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T01:10:39.000 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether stripping libraries is possible... yes 2024-04-25T01:10:39.000 INFO:teuthology.orchestra.run.smithi033.stdout:checking if libtool supports shared libraries... yes 2024-04-25T01:10:39.000 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether to build shared libraries... yes 2024-04-25T01:10:39.000 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether to build static libraries... yes 2024-04-25T01:10:39.010 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gcc... (cached) gcc 2024-04-25T01:10:39.037 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-04-25T01:10:39.037 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T01:10:39.037 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-04-25T01:10:39.037 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T01:10:39.037 INFO:teuthology.orchestra.run.smithi033.stderr:./configure: line 12753: test: $(SHELL): binary operator expected 2024-04-25T01:10:39.038 INFO:teuthology.orchestra.run.smithi033.stdout:checking for tar... /bin/tar 2024-04-25T01:10:39.038 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gzip... /bin/gzip 2024-04-25T01:10:39.049 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether gcc -MM is supported... yes 2024-04-25T01:10:39.050 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sort... /bin/sort 2024-04-25T01:10:39.050 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether ln -s works... yes 2024-04-25T01:10:39.050 INFO:teuthology.orchestra.run.smithi033.stdout:checking for rpm... no 2024-04-25T01:10:39.085 INFO:teuthology.orchestra.run.smithi033.stdout:checking for assert.h... yes 2024-04-25T01:10:39.127 INFO:teuthology.orchestra.run.smithi033.stdout:checking for bstring.h... no 2024-04-25T01:10:39.161 INFO:teuthology.orchestra.run.smithi033.stdout:checking for libgen.h... yes 2024-04-25T01:10:39.234 INFO:teuthology.orchestra.run.smithi033.stdout:checking for dirent.h... yes 2024-04-25T01:10:39.310 INFO:teuthology.orchestra.run.smithi033.stdout:checking for errno.h... yes 2024-04-25T01:10:39.349 INFO:teuthology.orchestra.run.smithi033.stdout:checking for malloc.h... yes 2024-04-25T01:10:39.383 INFO:teuthology.orchestra.run.smithi033.stdout:checking for uuid.h... no 2024-04-25T01:10:39.420 INFO:teuthology.orchestra.run.smithi033.stdout:checking for uuid/uuid.h... yes 2024-04-25T01:10:39.457 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/uuid.h... no 2024-04-25T01:10:39.493 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/file.h... yes 2024-04-25T01:10:39.527 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/fcntl.h... yes 2024-04-25T01:10:39.563 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/syssgi.h... no 2024-04-25T01:10:39.633 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/param.h... yes 2024-04-25T01:10:39.633 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T01:10:39.667 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/statvfs.h... yes 2024-04-25T01:10:39.705 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/time.h... yes 2024-04-25T01:10:39.740 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/ioctl.h... yes 2024-04-25T01:10:39.781 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/wait.h... yes 2024-04-25T01:10:39.782 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/types.h... (cached) yes 2024-04-25T01:10:39.782 INFO:teuthology.orchestra.run.smithi033.stdout:checking for strings.h... (cached) yes 2024-04-25T01:10:39.813 INFO:teuthology.orchestra.run.smithi033.stdout:checking for err.h... yes 2024-04-25T01:10:39.853 INFO:teuthology.orchestra.run.smithi033.stdout:checking for linux/falloc.h... yes 2024-04-25T01:10:39.889 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T01:10:39.923 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T01:10:39.963 INFO:teuthology.orchestra.run.smithi033.stdout:checking for xfs/platform_defs.h... no 2024-04-25T01:10:40.002 INFO:teuthology.orchestra.run.smithi033.stdout:checking for btrfs/ioctl.h... no 2024-04-25T01:10:40.036 INFO:teuthology.orchestra.run.smithi033.stdout:checking for cifs/ioctl.h... no 2024-04-25T01:10:40.069 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/mman.h... yes 2024-04-25T01:10:40.104 INFO:teuthology.orchestra.run.smithi033.stdout:checking for linux/ext4.h... no 2024-04-25T01:10:40.118 INFO:teuthology.orchestra.run.smithi033.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T01:10:40.165 INFO:teuthology.orchestra.run.smithi033.stdout:checking for uuid_compare... no 2024-04-25T01:10:40.201 INFO:teuthology.orchestra.run.smithi033.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T01:10:40.313 INFO:teuthology.orchestra.run.smithi033.stdout:checking for xfs/xfs.h... yes 2024-04-25T01:10:40.332 INFO:teuthology.orchestra.run.smithi033.stdout:checking for xfs/libxfs.h... no 2024-04-25T01:10:40.382 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gcc options needed to detect all undeclared functions... none needed 2024-04-25T01:10:40.403 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T01:10:40.450 INFO:teuthology.orchestra.run.smithi033.stdout:checking for xfs/xqm.h... yes 2024-04-25T01:10:40.517 INFO:teuthology.orchestra.run.smithi033.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T01:10:40.553 INFO:teuthology.orchestra.run.smithi033.stdout:checking for xfs/handle.h... yes 2024-04-25T01:10:40.589 INFO:teuthology.orchestra.run.smithi033.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T01:10:40.629 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/acl.h... yes 2024-04-25T01:10:40.670 INFO:teuthology.orchestra.run.smithi033.stdout:checking for acl/libacl.h... yes 2024-04-25T01:10:40.706 INFO:teuthology.orchestra.run.smithi033.stdout:checking for acl_init in -lacl... yes 2024-04-25T01:10:40.737 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gdbm-ndbm.h... no 2024-04-25T01:10:40.769 INFO:teuthology.orchestra.run.smithi033.stdout:checking for gdbm/ndbm.h... no 2024-04-25T01:10:40.802 INFO:teuthology.orchestra.run.smithi033.stdout:checking for ndbm.h... no 2024-04-25T01:10:40.836 INFO:teuthology.orchestra.run.smithi033.stdout:checking for libaio.h... yes 2024-04-25T01:10:40.884 INFO:teuthology.orchestra.run.smithi033.stdout:checking for liburing.h... yes 2024-04-25T01:10:40.921 INFO:teuthology.orchestra.run.smithi033.stdout:checking for sys/capability.h... yes 2024-04-25T01:10:40.953 INFO:teuthology.orchestra.run.smithi033.stdout:checking for linux/fiemap.h... yes 2024-04-25T01:10:40.988 INFO:teuthology.orchestra.run.smithi033.stdout:checking for fallocate... yes 2024-04-25T01:10:41.025 INFO:teuthology.orchestra.run.smithi033.stdout:checking for open_by_handle_at... yes 2024-04-25T01:10:41.057 INFO:teuthology.orchestra.run.smithi033.stdout:checking for linux/fs.h... yes 2024-04-25T01:10:41.087 INFO:teuthology.orchestra.run.smithi033.stdout:checking for btrfsutil.h... no 2024-04-25T01:10:41.132 INFO:teuthology.orchestra.run.smithi033.stdout:checking for copy_file_range... yes 2024-04-25T01:10:41.175 INFO:teuthology.orchestra.run.smithi033.stdout:checking for SEEK_DATA... yes 2024-04-25T01:10:41.239 INFO:teuthology.orchestra.run.smithi033.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T01:10:41.285 INFO:teuthology.orchestra.run.smithi033.stdout:checking for nftw... yes 2024-04-25T01:10:41.326 INFO:teuthology.orchestra.run.smithi033.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T01:10:41.371 INFO:teuthology.orchestra.run.smithi033.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T01:10:41.407 INFO:teuthology.orchestra.run.smithi033.stdout:checking for FICLONE... yes 2024-04-25T01:10:41.443 INFO:teuthology.orchestra.run.smithi033.stdout:checking for renameat2... yes 2024-04-25T01:10:41.479 INFO:teuthology.orchestra.run.smithi033.stdout:checking for reallocarray... yes 2024-04-25T01:10:41.512 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct mount_attr... yes 2024-04-25T01:10:41.553 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T01:10:41.587 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T01:10:41.627 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T01:10:41.666 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T01:10:41.709 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T01:10:41.751 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T01:10:41.792 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T01:10:41.826 INFO:teuthology.orchestra.run.smithi033.stdout:checking for linux/btrfs.h... yes 2024-04-25T01:10:41.869 INFO:teuthology.orchestra.run.smithi033.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T01:10:41.889 INFO:teuthology.orchestra.run.smithi033.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T01:10:41.917 INFO:teuthology.orchestra.run.smithi033.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T01:10:41.936 INFO:teuthology.orchestra.run.smithi033.stdout:configure: creating ./config.status 2024-04-25T01:10:42.326 INFO:teuthology.orchestra.run.smithi033.stdout:config.status: creating include/builddefs 2024-04-25T01:10:42.341 INFO:teuthology.orchestra.run.smithi033.stdout:config.status: creating include/config.h 2024-04-25T01:10:42.353 INFO:teuthology.orchestra.run.smithi033.stdout:config.status: executing libtool commands 2024-04-25T01:10:42.399 INFO:teuthology.orchestra.run.smithi033.stdout:Building include 2024-04-25T01:10:42.402 INFO:teuthology.orchestra.run.smithi033.stdout:Building lib 2024-04-25T01:10:42.406 INFO:teuthology.orchestra.run.smithi033.stdout: [LTDEP] 2024-04-25T01:10:42.564 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] dataascii.lo 2024-04-25T01:10:42.691 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] databin.lo 2024-04-25T01:10:42.846 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] datapid.lo 2024-04-25T01:10:42.960 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] file_lock.lo 2024-04-25T01:10:43.130 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] forker.lo 2024-04-25T01:10:43.281 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] pattern.lo 2024-04-25T01:10:43.401 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] open_flags.lo 2024-04-25T01:10:43.558 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] random_range.lo 2024-04-25T01:10:43.815 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] string_to_tokens.lo 2024-04-25T01:10:43.916 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] str_to_bytes.lo 2024-04-25T01:10:44.051 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] tlibio.lo 2024-04-25T01:10:44.315 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] write_log.lo 2024-04-25T01:10:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:write_log.c: In function 'wlog_record_write': 2024-04-25T01:10:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:write_log.c:205:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:44.379 INFO:teuthology.orchestra.run.smithi033.stderr: 205 | write(wfile->w_afd, wbuf, reclen); 2024-04-25T01:10:44.379 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:write_log.c:209:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:44.380 INFO:teuthology.orchestra.run.smithi033.stderr: 209 | write(wfile->w_rfd, wbuf, reclen); 2024-04-25T01:10:44.380 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:44.585 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] random.lo 2024-04-25T01:10:44.689 INFO:teuthology.orchestra.run.smithi033.stdout: [LD] 2024-04-25T01:10:44.770 INFO:teuthology.orchestra.run.smithi033.stdout:Building ltp 2024-04-25T01:10:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: [DEP] 2024-04-25T01:10:44.931 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] doio 2024-04-25T01:10:45.661 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fsstress 2024-04-25T01:10:47.226 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fsx 2024-04-25T01:10:48.026 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] growfiles 2024-04-25T01:10:48.594 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] iogen 2024-04-25T01:10:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:iogen.c: In function 'main': 2024-04-25T01:10:48.738 INFO:teuthology.orchestra.run.smithi033.stderr:iogen.c:280:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:48.738 INFO:teuthology.orchestra.run.smithi033.stderr: 280 | write(outfd, (char *)&req, sizeof(req)); 2024-04-25T01:10:48.738 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:iogen.c: In function 'parse_cmdline': 2024-04-25T01:10:48.741 INFO:teuthology.orchestra.run.smithi033.stderr:iogen.c:1535:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:48.741 INFO:teuthology.orchestra.run.smithi033.stderr: 1535 | getcwd(fptr->f_path, 2024-04-25T01:10:48.741 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:48.741 INFO:teuthology.orchestra.run.smithi033.stderr: 1536 | sizeof(fptr->f_path)-1); 2024-04-25T01:10:48.741 INFO:teuthology.orchestra.run.smithi033.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:49.027 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-stress 2024-04-25T01:10:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:aio-stress.c: In function 'setup_shared_mem': 2024-04-25T01:10:49.166 INFO:teuthology.orchestra.run.smithi033.stderr:aio-stress.c:1013:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:49.166 INFO:teuthology.orchestra.run.smithi033.stderr: 1013 | ftruncate(fd, total_ram); 2024-04-25T01:10:49.167 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:49.511 INFO:teuthology.orchestra.run.smithi033.stdout:Building src 2024-04-25T01:10:49.520 INFO:teuthology.orchestra.run.smithi033.stdout: [DEP] 2024-04-25T01:10:51.722 INFO:teuthology.orchestra.run.smithi033.stdout: [DEP] 2024-04-25T01:10:51.759 INFO:teuthology.orchestra.run.smithi033.stdout: [DEP] 2024-04-25T01:10:51.767 INFO:teuthology.orchestra.run.smithi033.stdout: [DEP] 2024-04-25T01:10:52.026 INFO:teuthology.orchestra.run.smithi033.stdout: [DEP] 2024-04-25T01:10:52.050 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] dirstress 2024-04-25T01:10:52.313 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fill 2024-04-25T01:10:52.478 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fill2 2024-04-25T01:10:52.716 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] getpagesize 2024-04-25T01:10:52.844 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] holes 2024-04-25T01:10:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] lstat64 2024-04-25T01:10:53.231 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] nametest 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:nametest.c: In function 'main': 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:nametest.c:236:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr: 236 | write(1, "\n", 1); 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:nametest.c:238:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr: 238 | write(1, ".", 1); 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr:nametest.c:288:33: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:53.361 INFO:teuthology.orchestra.run.smithi033.stderr: 288 | write(1, ".", 1); 2024-04-25T01:10:53.362 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:10:53.490 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] permname 2024-04-25T01:10:53.691 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] randholes 2024-04-25T01:10:53.950 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] runas 2024-04-25T01:10:54.124 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] truncfile 2024-04-25T01:10:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] usemem 2024-04-25T01:10:54.453 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] mmapcat 2024-04-25T01:10:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:mmapcat.c: In function 'main': 2024-04-25T01:10:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:mmapcat.c:29:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:54.574 INFO:teuthology.orchestra.run.smithi033.stderr: 29 | write(1,ptr2,ptr - ptr2); 2024-04-25T01:10:54.574 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:54.611 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] append_reader 2024-04-25T01:10:54.751 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] append_writer 2024-04-25T01:10:54.893 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] dirperf 2024-04-25T01:10:55.124 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] metaperf 2024-04-25T01:10:55.251 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c: In function 'crfiles': 2024-04-25T01:10:55.251 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c:166:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:55.251 INFO:teuthology.orchestra.run.smithi033.stderr: 166 | write(fd, buf, fsize); 2024-04-25T01:10:55.251 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:55.252 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c: In function 't_linkun': 2024-04-25T01:10:55.252 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c:454:25: warning: ignoring return value of 'link' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:55.252 INFO:teuthology.orchestra.run.smithi033.stderr: 454 | link("a", *fnp); 2024-04-25T01:10:55.252 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~ 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c: In function 't_chown': 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c:404:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr: 404 | chown(*fnp, 2, -1); 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c:406:33: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr: 406 | chown(*fnp, 1, -1); 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c: In function 'i_chown': 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c:291:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr: 291 | chown(*fnp, 1, -1); 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c: In function 'main': 2024-04-25T01:10:55.253 INFO:teuthology.orchestra.run.smithi033.stderr:metaperf.c:152:9: warning: ignoring return value of 'chdir' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:55.254 INFO:teuthology.orchestra.run.smithi033.stderr: 152 | chdir(".."); 2024-04-25T01:10:55.254 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~ 2024-04-25T01:10:55.428 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] devzero 2024-04-25T01:10:55.622 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] feature 2024-04-25T01:10:55.854 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] alloc 2024-04-25T01:10:55.983 INFO:teuthology.orchestra.run.smithi033.stderr:alloc.c: In function 'main': 2024-04-25T01:10:55.983 INFO:teuthology.orchestra.run.smithi033.stderr:alloc.c:247:17: warning: ignoring return value of 'mkstemp' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:55.983 INFO:teuthology.orchestra.run.smithi033.stderr: 247 | mkstemp(tmpfile); 2024-04-25T01:10:55.983 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:10:56.081 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fault 2024-04-25T01:10:56.258 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fstest 2024-04-25T01:10:56.508 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_access_root 2024-04-25T01:10:56.613 INFO:teuthology.orchestra.run.smithi033.stderr:t_access_root.c: In function 'main': 2024-04-25T01:10:56.613 INFO:teuthology.orchestra.run.smithi033.stderr:t_access_root.c:55:5: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:56.613 INFO:teuthology.orchestra.run.smithi033.stderr: 55 | system(cmd); 2024-04-25T01:10:56.614 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~ 2024-04-25T01:10:56.657 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] godown 2024-04-25T01:10:56.827 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] resvtest 2024-04-25T01:10:57.018 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] writemod 2024-04-25T01:10:57.163 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] writev_on_pagefault 2024-04-25T01:10:57.337 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] makeextents 2024-04-25T01:10:57.524 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] itrash 2024-04-25T01:10:57.682 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] rename 2024-04-25T01:10:57.806 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] multi_open_unlink 2024-04-25T01:10:57.979 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] unwritten_sync 2024-04-25T01:10:58.165 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] genhashnames 2024-04-25T01:10:58.362 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_holes 2024-04-25T01:10:58.514 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_writev 2024-04-25T01:10:58.678 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_truncate_cmtime 2024-04-25T01:10:58.833 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] dirhash_collide 2024-04-25T01:10:59.011 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_rename_overwrite 2024-04-25T01:10:59.140 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] holetest 2024-04-25T01:10:59.268 INFO:teuthology.orchestra.run.smithi033.stderr:holetest.c: In function 'pt_write_marker': 2024-04-25T01:10:59.268 INFO:teuthology.orchestra.run.smithi033.stderr:holetest.c:140:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:59.268 INFO:teuthology.orchestra.run.smithi033.stderr: 140 | pwrite(fd, &tid, sizeof(tid), i * page_size + pgoff); 2024-04-25T01:10:59.268 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:59.390 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_truncate_self 2024-04-25T01:10:59.527 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] af_unix 2024-04-25T01:10:59.670 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_stale_pmd 2024-04-25T01:10:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:t_mmap_stale_pmd.c: In function 'main': 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr:t_mmap_stale_pmd.c:42:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr: 42 | ftruncate(fd, 0); 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr:t_mmap_stale_pmd.c:43:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr: 43 | ftruncate(fd, MiB(4)); 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr:t_mmap_stale_pmd.c:55:9: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr: 55 | pwrite(fd, buffer, strlen(buffer), MiB(2)); 2024-04-25T01:10:59.785 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:59.822 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_cow_race 2024-04-25T01:10:59.943 INFO:teuthology.orchestra.run.smithi033.stderr:t_mmap_cow_race.c: In function 'main': 2024-04-25T01:10:59.944 INFO:teuthology.orchestra.run.smithi033.stderr:t_mmap_cow_race.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:59.944 INFO:teuthology.orchestra.run.smithi033.stderr: 61 | ftruncate(fd, 0); 2024-04-25T01:10:59.944 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:10:59.944 INFO:teuthology.orchestra.run.smithi033.stderr:t_mmap_cow_race.c:62:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:10:59.944 INFO:teuthology.orchestra.run.smithi033.stderr: 62 | ftruncate(fd, MiB(2)); 2024-04-25T01:10:59.944 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:10:59.990 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_fallocate 2024-04-25T01:11:00.128 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fsync-err 2024-04-25T01:11:00.310 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_write_ro 2024-04-25T01:11:00.431 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:00.433 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:00.457 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T01:11:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:t_ext4_dax_journal_corruption.c: In function 'main': 2024-04-25T01:11:00.563 INFO:teuthology.orchestra.run.smithi033.stderr:t_ext4_dax_journal_corruption.c:61:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:00.563 INFO:teuthology.orchestra.run.smithi033.stderr: 61 | ftruncate(fd, 0); 2024-04-25T01:11:00.563 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:11:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T01:11:00.769 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_ofd_locks 2024-04-25T01:11:00.976 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_collision 2024-04-25T01:11:01.202 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] mmap-write-concurrent 2024-04-25T01:11:01.379 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_get_file_time 2024-04-25T01:11:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_create_short_dirs 2024-04-25T01:11:01.686 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_create_long_dirs 2024-04-25T01:11:01.857 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_enospc 2024-04-25T01:11:02.054 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_writev_overlap 2024-04-25T01:11:02.237 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] checkpoint_journal 2024-04-25T01:11:02.385 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] mmap-rw-fault 2024-04-25T01:11:02.565 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] allocstale 2024-04-25T01:11:02.729 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T01:11:02.901 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fake-dump-rootino 2024-04-25T01:11:03.046 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] dio-buf-fault 2024-04-25T01:11:03.194 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] rewinddir-test 2024-04-25T01:11:03.368 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] readdir-while-renames 2024-04-25T01:11:03.544 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] xfsctl 2024-04-25T01:11:03.733 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] bstat 2024-04-25T01:11:03.951 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mtab 2024-04-25T01:11:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:t_mtab.c: In function 'update_mtab': 2024-04-25T01:11:04.067 INFO:teuthology.orchestra.run.smithi033.stderr:t_mtab.c:256:17: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 256 | chown (mounted_temp, sbuf.st_uid, sbuf.st_gid); 2024-04-25T01:11:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] getdevicesize 2024-04-25T01:11:04.292 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T01:11:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:In file included from preallo_rw_pattern_reader.c:2: 2024-04-25T01:11:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:iopat.c: In function 'main': 2024-04-25T01:11:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:iopat.c:38:4: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:04.420 INFO:teuthology.orchestra.run.smithi033.stderr: 38 | read(fd, &x, 1048576); 2024-04-25T01:11:04.420 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:04.458 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T01:11:04.585 INFO:teuthology.orchestra.run.smithi033.stderr:In file included from preallo_rw_pattern_writer.c:2: 2024-04-25T01:11:04.585 INFO:teuthology.orchestra.run.smithi033.stderr:iopat.c: In function 'main': 2024-04-25T01:11:04.585 INFO:teuthology.orchestra.run.smithi033.stderr:iopat.c:34:4: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:04.585 INFO:teuthology.orchestra.run.smithi033.stderr: 34 | write(fd, &x, 1048576); 2024-04-25T01:11:04.585 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:04.625 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] ftrunc 2024-04-25T01:11:04.779 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] trunc 2024-04-25T01:11:04.951 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fs_perms 2024-04-25T01:11:05.061 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c: In function 'main': 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c:54:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr: 54 | system("rm -f test.file"); 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c: In function 'testsetup': 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c:67:4: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr: 67 | system("cp testx.file test.file"); 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c:69:4: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr: 69 | chown("test.file",cuserId,cgroupId); 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c: In function 'testfperm': 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c:123:5: warning: ignoring return value of 'seteuid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:05.062 INFO:teuthology.orchestra.run.smithi033.stderr: 123 | seteuid(0); 2024-04-25T01:11:05.063 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~ 2024-04-25T01:11:05.063 INFO:teuthology.orchestra.run.smithi033.stderr:fs_perms.c:124:5: warning: ignoring return value of 'setegid' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:05.063 INFO:teuthology.orchestra.run.smithi033.stderr: 124 | setegid(0); 2024-04-25T01:11:05.063 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~ 2024-04-25T01:11:05.113 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] testx 2024-04-25T01:11:05.228 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] looptest 2024-04-25T01:11:05.418 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] locktest 2024-04-25T01:11:05.757 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] unwritten_mmap 2024-04-25T01:11:05.930 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] bulkstat_unlink_test 2024-04-25T01:11:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:bulkstat_unlink_test.c: In function 'main': 2024-04-25T01:11:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:bulkstat_unlink_test.c:106:25: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:06.050 INFO:teuthology.orchestra.run.smithi033.stderr: 106 | write(fd[i], fname, sizeof(fname)); 2024-04-25T01:11:06.050 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:06.050 INFO:teuthology.orchestra.run.smithi033.stderr:bulkstat_unlink_test.c:178:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:06.050 INFO:teuthology.orchestra.run.smithi033.stderr: 178 | system(fname); 2024-04-25T01:11:06.050 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~ 2024-04-25T01:11:06.138 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] deduperace 2024-04-25T01:11:06.348 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T01:11:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:bulkstat_unlink_test_modified.c: In function 'main': 2024-04-25T01:11:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:bulkstat_unlink_test_modified.c:83:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:06.472 INFO:teuthology.orchestra.run.smithi033.stderr: 83 | write(fd, fname, sizeof(fname)); 2024-04-25T01:11:06.472 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:06.472 INFO:teuthology.orchestra.run.smithi033.stderr:bulkstat_unlink_test_modified.c:139:13: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:06.473 INFO:teuthology.orchestra.run.smithi033.stderr: 139 | write(fd, fname, sizeof(fname)); 2024-04-25T01:11:06.473 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:06.473 INFO:teuthology.orchestra.run.smithi033.stderr:bulkstat_unlink_test_modified.c:185:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:06.473 INFO:teuthology.orchestra.run.smithi033.stderr: 185 | system(fname); 2024-04-25T01:11:06.473 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~ 2024-04-25T01:11:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_dir_offset 2024-04-25T01:11:06.940 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_futimens 2024-04-25T01:11:07.085 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_immutable 2024-04-25T01:11:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c: In function 'create_test_area': 2024-04-25T01:11:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2061:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.222 INFO:teuthology.orchestra.run.smithi033.stderr: 2061 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T01:11:07.222 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2096:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.223 INFO:teuthology.orchestra.run.smithi033.stderr: 2096 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T01:11:07.223 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c: In function 'remove_test_area': 2024-04-25T01:11:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2152:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.223 INFO:teuthology.orchestra.run.smithi033.stderr: 2152 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2163:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr: 2163 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2175:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr: 2175 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2186:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr: 2186 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-04-25T01:11:07.224 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.226 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2197:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.226 INFO:teuthology.orchestra.run.smithi033.stderr: 2197 | asprintf(&path, "%s/append-only.f", dir); 2024-04-25T01:11:07.226 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.226 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:2209:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.226 INFO:teuthology.orchestra.run.smithi033.stderr: 2209 | asprintf(&path, "%s/immutable.f", dir); 2024-04-25T01:11:07.226 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.226 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c: In function 'check_test_area': 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1788:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr: 1788 | asprintf(&path, "%s/", dir); 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1800:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr: 1800 | asprintf(&path, "%s/immutable.f", dir); 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1812:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr: 1812 | asprintf(&path, "%s/append-only.f", dir); 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.227 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1824:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: 1824 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1837:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: 1837 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1850:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: 1850 | asprintf(&path, "%s/immutable.d/file", dir); 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1862:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: 1862 | asprintf(&path, "%s/append-only.d/file", dir); 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1874:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: 1874 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-04-25T01:11:07.228 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.229 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1887:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.229 INFO:teuthology.orchestra.run.smithi033.stderr: 1887 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-04-25T01:11:07.229 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c: In function 'test_immutable': 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:151:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr: 151 | asprintf(&path, "%s/immutable.f", dir); 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:381:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr: 381 | asprintf(&linkpath, "%s/immutable.f.hardlink", dir); 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:408:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr: 408 | chown(path, 0, 0); 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:485:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.231 INFO:teuthology.orchestra.run.smithi033.stderr: 485 | asprintf(&linkpath, "%s/immutable.f.newname", dir); 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:532:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: 532 | asprintf(&path, "%s/immutable.d/file", dir); 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:553:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: 553 | chown(path, 0, 0); 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:556:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: 556 | asprintf(&linkpath, "%s/immutable.d/file.link", dir); 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:575:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: 575 | asprintf(&linkpath, "%s/immutable.d/file.newname", dir); 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.232 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:595:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: 595 | asprintf(&path, "%s/immutable.d/newfile", dir); 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:619:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: 619 | asprintf(&path, "%s/immutable.d/newdir", dir); 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:631:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: 631 | asprintf(&path, "%s/immutable.d/dir/newfile-%d", dir, getuid()); 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:653:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: 653 | chown(path, 0, 0); 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:657:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: 657 | asprintf(&path, "%s/immutable.d/dir", dir); 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.233 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:668:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr: 668 | asprintf(&path, "%s/immutable.d", dir); 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:705:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr: 705 | chown(path, 0, 0); 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:782:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr: 782 | asprintf(&path, "%s/empty-immutable.d", dir); 2024-04-25T01:11:07.234 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c: In function 'test_append': 2024-04-25T01:11:07.236 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:822:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.236 INFO:teuthology.orchestra.run.smithi033.stderr: 822 | asprintf(&path, "%s/append-only.f", dir); 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1024:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: 1024 | asprintf(&linkpath, "%s/append-only.f.hardlink", dir); 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1051:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: 1051 | chown(path, 0, 0); 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1126:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: 1126 | asprintf(&linkpath, "%s/append-only.f.newname", dir); 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1495:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: 1495 | asprintf(&path, "%s/append-only.d/file", dir); 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.237 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1516:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: 1516 | chown(path, 0, 0); 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1519:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: 1519 | asprintf(&linkpath, "%s/append-only.d/file.link-%d", dir, getuid()); 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1529:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: 1529 | asprintf(&linkpath, "%s/append-only.d/file.symlink-%d", dir, getuid()); 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1539:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: 1539 | asprintf(&linkpath, "%s/append-only.d/file.newname", dir); 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1559:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: 1559 | asprintf(&path, "%s/append-only.d/newfile-%d", dir, getuid()); 2024-04-25T01:11:07.238 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1573:11: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr: 1573 | asprintf(&path, "%s/append-only.d/newdev-%d", dir, getuid()); 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1586:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr: 1586 | asprintf(&path, "%s/append-only.d/newdir-%d", dir, getuid()); 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1596:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr: 1596 | asprintf(&path, "%s/append-only.d/newdir-%d/newfile-%d", dir, getuid(), getuid()); 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.239 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1618:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: 1618 | chown(path, 0, 0); 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1622:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: 1622 | asprintf(&path, "%s/append-only.d/dir/newfile-%d", dir, getuid()); 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1644:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: 1644 | chown(path, 0, 0); 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1648:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: 1648 | asprintf(&path, "%s/append-only.d/dir", dir); 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1659:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: 1659 | asprintf(&path, "%s/append-only.d", dir); 2024-04-25T01:11:07.240 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1692:16: warning: ignoring return value of 'chown' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.241 INFO:teuthology.orchestra.run.smithi033.stderr: 1692 | chown(path, 0, 0); 2024-04-25T01:11:07.241 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.241 INFO:teuthology.orchestra.run.smithi033.stderr:t_immutable.c:1769:6: warning: ignoring return value of 'asprintf' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.241 INFO:teuthology.orchestra.run.smithi033.stderr: 1769 | asprintf(&path, "%s/empty-append-only.d", dir); 2024-04-25T01:11:07.241 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.813 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] stale_handle 2024-04-25T01:11:07.928 INFO:teuthology.orchestra.run.smithi033.stderr:stale_handle.c: In function 'main': 2024-04-25T01:11:07.928 INFO:teuthology.orchestra.run.smithi033.stderr:stale_handle.c:102:9: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:07.929 INFO:teuthology.orchestra.run.smithi033.stderr: 102 | system("echo 3 > /proc/sys/vm/drop_caches"); 2024-04-25T01:11:07.929 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:07.979 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] pwrite_mmap_blocked 2024-04-25T01:11:08.118 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_dir_offset2 2024-04-25T01:11:08.318 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] seek_sanity_test 2024-04-25T01:11:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c: In function 'test18': 2024-04-25T01:11:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c:460:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:08.438 INFO:teuthology.orchestra.run.smithi033.stderr: 460 | ftruncate(fd, 0); 2024-04-25T01:11:08.438 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:11:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c: In function 'test_basic_support': 2024-04-25T01:11:08.441 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c:1207:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr: 1207 | ftruncate(fd, 0); 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c:1256:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr: 1256 | ftruncate(fd, 0); 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c: In function 'get_io_sizes': 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c:88:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:08.442 INFO:teuthology.orchestra.run.smithi033.stderr: 88 | ftruncate(fd, 0); 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c:89:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr: 89 | pwrite(fd, "a", 1, offset); 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c:98:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr: 98 | ftruncate(fd, 0); 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~ 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr:seek_sanity_test.c:99:17: warning: ignoring return value of 'pwrite' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr: 99 | pwrite(fd, "a", 1, offset); 2024-04-25T01:11:08.443 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:08.778 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] seek_copy_test 2024-04-25T01:11:08.968 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_readdir_1 2024-04-25T01:11:09.086 INFO:teuthology.orchestra.run.smithi033.stderr:t_readdir_1.c: In function 'main': 2024-04-25T01:11:09.086 INFO:teuthology.orchestra.run.smithi033.stderr:t_readdir_1.c:30:25: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:09.086 INFO:teuthology.orchestra.run.smithi033.stderr: 30 | read(fd, buf, 100); 2024-04-25T01:11:09.086 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:09.125 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_readdir_2 2024-04-25T01:11:09.284 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fsync-tester 2024-04-25T01:11:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:fsync-tester.c: In function 'main': 2024-04-25T01:11:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:fsync-tester.c:391:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:09.401 INFO:teuthology.orchestra.run.smithi033.stderr: 391 | read(fd, &seed, sizeof(seed)); 2024-04-25T01:11:09.401 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:09.401 INFO:teuthology.orchestra.run.smithi033.stderr:fsync-tester.c:541:17: warning: ignoring return value of 'system' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:09.401 INFO:teuthology.orchestra.run.smithi033.stderr: 541 | system("reboot -fn"); 2024-04-25T01:11:09.402 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:fsync-tester.c: In function 'drop_all_caches': 2024-04-25T01:11:09.402 INFO:teuthology.orchestra.run.smithi033.stderr:fsync-tester.c:90:9: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:09.402 INFO:teuthology.orchestra.run.smithi033.stderr: 90 | write(fd, value, sizeof(value)-1); 2024-04-25T01:11:09.402 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:09.559 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] nsexec 2024-04-25T01:11:09.759 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] cloner 2024-04-25T01:11:09.941 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] renameat2 2024-04-25T01:11:10.098 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_getcwd 2024-04-25T01:11:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:t_getcwd.c: In function 'test_getcwd': 2024-04-25T01:11:10.222 INFO:teuthology.orchestra.run.smithi033.stderr:t_getcwd.c:21:17: warning: ignoring return value of 'getcwd' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:10.222 INFO:teuthology.orchestra.run.smithi033.stderr: 21 | getcwd(cur_cwd, BUF_SIZE); 2024-04-25T01:11:10.222 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:10.273 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] e4compact 2024-04-25T01:11:10.495 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] test-nextquota 2024-04-25T01:11:10.693 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] punch-alternating 2024-04-25T01:11:10.865 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T01:11:11.039 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] listxattr 2024-04-25T01:11:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] dio-interleaved 2024-04-25T01:11:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_dir_type 2024-04-25T01:11:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] dio-invalidate-cache 2024-04-25T01:11:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:dio-invalidate-cache.c: In function 'main': 2024-04-25T01:11:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:dio-invalidate-cache.c:286:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:11.632 INFO:teuthology.orchestra.run.smithi033.stderr: 286 | write(fd, buf, blksz * nr_child); 2024-04-25T01:11:11.632 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] stat_test 2024-04-25T01:11:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T01:11:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] attr_replace_test 2024-04-25T01:11:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] swapon 2024-04-25T01:11:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] mkswap 2024-04-25T01:11:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_attr_corruption 2024-04-25T01:11:12.848 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_open_tmpfiles 2024-04-25T01:11:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fscrypt-crypt-util 2024-04-25T01:11:13.698 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] bulkstat_null_ocount 2024-04-25T01:11:13.846 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] splice-test 2024-04-25T01:11:14.033 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] chprojid_fail 2024-04-25T01:11:14.211 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] detached_mounts_propagation 2024-04-25T01:11:14.404 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] ext4_resize 2024-04-25T01:11:14.553 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_readdir_3 2024-04-25T01:11:14.747 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] splice2pipe 2024-04-25T01:11:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:splice2pipe.c: In function 'prepare_pipe': 2024-04-25T01:11:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:splice2pipe.c:57:17: warning: ignoring return value of 'write' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:14.867 INFO:teuthology.orchestra.run.smithi033.stderr: 57 | write(p[1], buffer, n); 2024-04-25T01:11:14.867 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:14.867 INFO:teuthology.orchestra.run.smithi033.stderr:splice2pipe.c:65:17: warning: ignoring return value of 'read' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:14.867 INFO:teuthology.orchestra.run.smithi033.stderr: 65 | read(p[0], buffer, n); 2024-04-25T01:11:14.867 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:14.918 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] uuid_ioctl 2024-04-25T01:11:15.077 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T01:11:15.245 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fiemap-fault 2024-04-25T01:11:15.393 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fiemap-tester 2024-04-25T01:11:15.671 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_stripealign 2024-04-25T01:11:15.848 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] open_by_handle 2024-04-25T01:11:16.124 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_mmap_dio 2024-04-25T01:11:16.291 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] uring_read_fault 2024-04-25T01:11:16.483 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] xfsfind 2024-04-25T01:11:16.701 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] t_reflink_read_race 2024-04-25T01:11:16.900 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] fssum 2024-04-25T01:11:17.373 INFO:teuthology.orchestra.run.smithi033.stdout:Building log-writes 2024-04-25T01:11:17.381 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] replay-log 2024-04-25T01:11:17.734 INFO:teuthology.orchestra.run.smithi033.stdout:Building perf 2024-04-25T01:11:17.739 INFO:teuthology.orchestra.run.smithi033.stdout:Building aio-dio-regress 2024-04-25T01:11:17.747 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aiocp 2024-04-25T01:11:17.992 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T01:11:18.177 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T01:11:18.361 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-cow-race 2024-04-25T01:11:18.515 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-cycle-write 2024-04-25T01:11:18.618 INFO:teuthology.orchestra.run.smithi033.stderr:aio-dio-cycle-write.c: In function 'init_test': 2024-04-25T01:11:18.619 INFO:teuthology.orchestra.run.smithi033.stderr:aio-dio-cycle-write.c:83:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:18.619 INFO:teuthology.orchestra.run.smithi033.stderr: 83 | ftruncate(fd, buf_size); 2024-04-25T01:11:18.619 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:18.710 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-eof-race 2024-04-25T01:11:18.917 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-extend-stat 2024-04-25T01:11:19.092 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-fcntl-race 2024-04-25T01:11:19.258 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T01:11:19.409 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T01:11:19.572 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T01:11:19.748 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aiodio_sparse2 2024-04-25T01:11:19.855 INFO:teuthology.orchestra.run.smithi033.stderr:aiodio_sparse2.c: In function 'aiodio_sparse': 2024-04-25T01:11:19.856 INFO:teuthology.orchestra.run.smithi033.stderr:aiodio_sparse2.c:111:17: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: 111 | ftruncate(fd, filesize); 2024-04-25T01:11:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:19.856 INFO:teuthology.orchestra.run.smithi033.stderr:aiodio_sparse2.c: In function 'dirty_freeblocks': 2024-04-25T01:11:19.856 INFO:teuthology.orchestra.run.smithi033.stderr:aiodio_sparse2.c:274:9: warning: ignoring return value of 'ftruncate' declared with attribute 'warn_unused_result' [-Wunused-result] 2024-04-25T01:11:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: 274 | ftruncate(fd, size); 2024-04-25T01:11:19.857 INFO:teuthology.orchestra.run.smithi033.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-04-25T01:11:19.985 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T01:11:20.146 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-dio-write-verify 2024-04-25T01:11:20.367 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T01:11:20.511 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T01:11:20.654 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T01:11:20.841 INFO:teuthology.orchestra.run.smithi033.stdout:Building vfs 2024-04-25T01:11:20.850 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] vfstest 2024-04-25T01:11:25.153 INFO:teuthology.orchestra.run.smithi033.stdout: [CC] mount-idmapped 2024-04-25T01:11:25.584 INFO:teuthology.orchestra.run.smithi033.stdout:Building m4 2024-04-25T01:11:25.589 INFO:teuthology.orchestra.run.smithi033.stdout:Building common 2024-04-25T01:11:25.593 INFO:teuthology.orchestra.run.smithi033.stdout:Building tests 2024-04-25T01:11:25.599 INFO:teuthology.orchestra.run.smithi033.stdout:Building btrfs 2024-04-25T01:11:25.606 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/btrfs/group.list 2024-04-25T01:11:25.678 INFO:teuthology.orchestra.run.smithi033.stdout:Building ceph 2024-04-25T01:11:25.684 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ceph/group.list 2024-04-25T01:11:25.702 INFO:teuthology.orchestra.run.smithi033.stdout:Building cifs 2024-04-25T01:11:25.708 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/cifs/group.list 2024-04-25T01:11:25.724 INFO:teuthology.orchestra.run.smithi033.stdout:Building ext4 2024-04-25T01:11:25.730 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ext4/group.list 2024-04-25T01:11:25.778 INFO:teuthology.orchestra.run.smithi033.stdout:Building f2fs 2024-04-25T01:11:25.784 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/f2fs/group.list 2024-04-25T01:11:25.806 INFO:teuthology.orchestra.run.smithi033.stdout:Building generic 2024-04-25T01:11:25.815 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/generic/group.list 2024-04-25T01:11:25.938 INFO:teuthology.orchestra.run.smithi033.stdout:Building nfs 2024-04-25T01:11:25.945 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/nfs/group.list 2024-04-25T01:11:25.963 INFO:teuthology.orchestra.run.smithi033.stdout:Building ocfs2 2024-04-25T01:11:25.970 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ocfs2/group.list 2024-04-25T01:11:25.987 INFO:teuthology.orchestra.run.smithi033.stdout:Building overlay 2024-04-25T01:11:25.993 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/overlay/group.list 2024-04-25T01:11:26.043 INFO:teuthology.orchestra.run.smithi033.stdout:Building perf 2024-04-25T01:11:26.050 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/perf/group.list 2024-04-25T01:11:26.064 INFO:teuthology.orchestra.run.smithi033.stdout:Building selftest 2024-04-25T01:11:26.071 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/selftest/group.list 2024-04-25T01:11:26.086 INFO:teuthology.orchestra.run.smithi033.stdout:Building shared 2024-04-25T01:11:26.092 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/shared/group.list 2024-04-25T01:11:26.112 INFO:teuthology.orchestra.run.smithi033.stdout:Building tmpfs 2024-04-25T01:11:26.118 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/tmpfs/group.list 2024-04-25T01:11:26.134 INFO:teuthology.orchestra.run.smithi033.stdout:Building udf 2024-04-25T01:11:26.140 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/udf/group.list 2024-04-25T01:11:26.153 INFO:teuthology.orchestra.run.smithi033.stdout:Building xfs 2024-04-25T01:11:26.160 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/xfs/group.list 2024-04-25T01:11:26.273 DEBUG:teuthology.orchestra.run.smithi033:> (cd /tmp/tmp.mSBFkuSi7xxfstests-dev && exec sudo make install) 2024-04-25T01:11:26.300 INFO:teuthology.orchestra.run.smithi033.stdout:Building include 2024-04-25T01:11:26.303 INFO:teuthology.orchestra.run.smithi033.stdout:Building lib 2024-04-25T01:11:26.308 INFO:teuthology.orchestra.run.smithi033.stdout:Building ltp 2024-04-25T01:11:26.312 INFO:teuthology.orchestra.run.smithi033.stdout:Building src 2024-04-25T01:11:26.349 INFO:teuthology.orchestra.run.smithi033.stdout:Building log-writes 2024-04-25T01:11:26.353 INFO:teuthology.orchestra.run.smithi033.stdout:Building perf 2024-04-25T01:11:26.357 INFO:teuthology.orchestra.run.smithi033.stdout:Building aio-dio-regress 2024-04-25T01:11:26.362 INFO:teuthology.orchestra.run.smithi033.stdout:Building vfs 2024-04-25T01:11:26.366 INFO:teuthology.orchestra.run.smithi033.stdout:Building m4 2024-04-25T01:11:26.370 INFO:teuthology.orchestra.run.smithi033.stdout:Building common 2024-04-25T01:11:26.373 INFO:teuthology.orchestra.run.smithi033.stdout:Building tests 2024-04-25T01:11:26.378 INFO:teuthology.orchestra.run.smithi033.stdout:Building btrfs 2024-04-25T01:11:26.385 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/btrfs/group.list 2024-04-25T01:11:26.451 INFO:teuthology.orchestra.run.smithi033.stdout:Building ceph 2024-04-25T01:11:26.457 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ceph/group.list 2024-04-25T01:11:26.473 INFO:teuthology.orchestra.run.smithi033.stdout:Building cifs 2024-04-25T01:11:26.479 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/cifs/group.list 2024-04-25T01:11:26.493 INFO:teuthology.orchestra.run.smithi033.stdout:Building ext4 2024-04-25T01:11:26.498 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ext4/group.list 2024-04-25T01:11:26.538 INFO:teuthology.orchestra.run.smithi033.stdout:Building f2fs 2024-04-25T01:11:26.544 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/f2fs/group.list 2024-04-25T01:11:26.564 INFO:teuthology.orchestra.run.smithi033.stdout:Building generic 2024-04-25T01:11:26.571 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/generic/group.list 2024-04-25T01:11:26.674 INFO:teuthology.orchestra.run.smithi033.stdout:Building nfs 2024-04-25T01:11:26.679 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/nfs/group.list 2024-04-25T01:11:26.694 INFO:teuthology.orchestra.run.smithi033.stdout:Building ocfs2 2024-04-25T01:11:26.701 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ocfs2/group.list 2024-04-25T01:11:26.715 INFO:teuthology.orchestra.run.smithi033.stdout:Building overlay 2024-04-25T01:11:26.722 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/overlay/group.list 2024-04-25T01:11:26.765 INFO:teuthology.orchestra.run.smithi033.stdout:Building perf 2024-04-25T01:11:26.770 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/perf/group.list 2024-04-25T01:11:26.782 INFO:teuthology.orchestra.run.smithi033.stdout:Building selftest 2024-04-25T01:11:26.788 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/selftest/group.list 2024-04-25T01:11:26.802 INFO:teuthology.orchestra.run.smithi033.stdout:Building shared 2024-04-25T01:11:26.808 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/shared/group.list 2024-04-25T01:11:26.829 INFO:teuthology.orchestra.run.smithi033.stdout:Building tmpfs 2024-04-25T01:11:26.835 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/tmpfs/group.list 2024-04-25T01:11:26.851 INFO:teuthology.orchestra.run.smithi033.stdout:Building udf 2024-04-25T01:11:26.857 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/udf/group.list 2024-04-25T01:11:26.872 INFO:teuthology.orchestra.run.smithi033.stdout:Building xfs 2024-04-25T01:11:26.880 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/xfs/group.list 2024-04-25T01:11:27.003 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-25T01:11:27.005 INFO:teuthology.orchestra.run.smithi033.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T01:11:27.005 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-25T01:11:27.010 INFO:teuthology.orchestra.run.smithi033.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T01:11:27.010 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-25T01:11:27.013 INFO:teuthology.orchestra.run.smithi033.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-25T01:11:27.025 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-25T01:11:27.122 INFO:teuthology.orchestra.run.smithi033.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-25T01:11:27.132 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-25T01:11:27.156 INFO:teuthology.orchestra.run.smithi033.stdout:Building log-writes 2024-04-25T01:11:27.160 INFO:teuthology.orchestra.run.smithi033.stdout:Building perf 2024-04-25T01:11:27.164 INFO:teuthology.orchestra.run.smithi033.stdout:Building aio-dio-regress 2024-04-25T01:11:27.169 INFO:teuthology.orchestra.run.smithi033.stdout:Building vfs 2024-04-25T01:11:27.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake -C log-writes install 2024-04-25T01:11:27.174 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-25T01:11:27.184 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-25T01:11:27.196 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake -C perf install 2024-04-25T01:11:27.198 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-25T01:11:27.209 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-25T01:11:27.233 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-25T01:11:27.236 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-25T01:11:27.246 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-25T01:11:27.304 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake -C vfs install 2024-04-25T01:11:27.307 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-25T01:11:27.319 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-25T01:11:27.334 INFO:teuthology.orchestra.run.smithi033.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-25T01:11:27.346 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle t_mmap_dio uring_read_fault xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-25T01:11:29.103 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-25T01:11:29.239 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/bash ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-25T01:11:29.270 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-25T01:11:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-25T01:11:29.273 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-25T01:11:29.276 INFO:teuthology.orchestra.run.smithi033.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-25T01:11:29.286 INFO:teuthology.orchestra.run.smithi033.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-25T01:11:29.440 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-25T01:11:29.443 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/btrfs/ install 2024-04-25T01:11:29.448 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/btrfs/group.list 2024-04-25T01:11:29.518 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-25T01:11:29.529 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-25T01:11:30.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:30.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:11:30.585 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-25T01:11:30.597 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-25T01:11:31.641 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ceph/ install 2024-04-25T01:11:31.646 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ceph/group.list 2024-04-25T01:11:31.664 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-25T01:11:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-25T01:11:31.700 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-25T01:11:31.711 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-25T01:11:31.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/cifs/ install 2024-04-25T01:11:31.738 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/cifs/group.list 2024-04-25T01:11:31.754 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-25T01:11:31.766 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-25T01:11:31.779 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-25T01:11:31.790 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-25T01:11:31.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ext4/ install 2024-04-25T01:11:31.806 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ext4/group.list 2024-04-25T01:11:31.854 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-25T01:11:31.866 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-25T01:11:32.087 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-25T01:11:32.099 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-25T01:11:32.339 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/f2fs/ install 2024-04-25T01:11:32.344 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/f2fs/group.list 2024-04-25T01:11:32.366 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-25T01:11:32.379 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-25T01:11:32.394 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-25T01:11:32.407 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-25T01:11:32.423 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/generic/ install 2024-04-25T01:11:32.429 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/generic/group.list 2024-04-25T01:11:32.553 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-25T01:11:32.565 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-25T01:11:35.132 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-25T01:11:35.145 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-25T01:11:37.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/nfs/ install 2024-04-25T01:11:37.755 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/nfs/group.list 2024-04-25T01:11:37.774 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-25T01:11:37.787 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-25T01:11:37.799 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-25T01:11:37.812 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-25T01:11:37.824 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ocfs2/ install 2024-04-25T01:11:37.830 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/ocfs2/group.list 2024-04-25T01:11:37.847 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-25T01:11:37.860 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-25T01:11:37.872 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-25T01:11:37.884 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-25T01:11:37.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/overlay/ install 2024-04-25T01:11:37.900 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/overlay/group.list 2024-04-25T01:11:37.952 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-25T01:11:37.964 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-25T01:11:38.310 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-25T01:11:38.324 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-25T01:11:38.678 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/perf/ install 2024-04-25T01:11:38.683 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/perf/group.list 2024-04-25T01:11:38.696 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-25T01:11:38.709 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-25T01:11:38.720 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-25T01:11:38.732 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-25T01:11:38.745 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/selftest/ install 2024-04-25T01:11:38.749 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/selftest/group.list 2024-04-25T01:11:38.764 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-25T01:11:38.776 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-25T01:11:38.804 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-25T01:11:38.815 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-25T01:11:38.841 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/shared/ install 2024-04-25T01:11:38.846 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/shared/group.list 2024-04-25T01:11:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-25T01:11:38.880 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-25T01:11:38.897 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-25T01:11:38.908 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-25T01:11:38.925 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/tmpfs/ install 2024-04-25T01:11:38.930 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/tmpfs/group.list 2024-04-25T01:11:38.945 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-25T01:11:38.956 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-25T01:11:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-25T01:11:38.977 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-25T01:11:38.989 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/udf/ install 2024-04-25T01:11:38.992 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/udf/group.list 2024-04-25T01:11:39.005 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-25T01:11:39.016 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-25T01:11:39.027 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-25T01:11:39.038 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-25T01:11:39.049 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/xfs/ install 2024-04-25T01:11:39.054 INFO:teuthology.orchestra.run.smithi033.stdout: [GROUP] /tmp/tmp.mSBFkuSi7xxfstests-dev/tests/xfs/group.list 2024-04-25T01:11:39.173 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-25T01:11:39.185 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-25T01:11:41.558 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-25T01:11:41.570 INFO:teuthology.orchestra.run.smithi033.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-25T01:11:43.919 INFO:teuthology.orchestra.run.smithi033.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-25T01:11:43.931 INFO:teuthology.orchestra.run.smithi033.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-25T01:11:43.943 INFO:teuthology.orchestra.run.smithi033.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-25T01:11:43.955 INFO:teuthology.orchestra.run.smithi033.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-25T01:11:43.969 INFO:tasks.cephfs.test_acls:client is fuse mounted 2024-04-25T01:11:43.969 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T01:11:43.969 DEBUG:teuthology.orchestra.run.smithi033:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.mSBFkuSi7xxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-25T01:11:44.214 INFO:teuthology.orchestra.run.smithi033.stdout:FSTYP -- ceph-fuse 2024-04-25T01:11:44.219 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM -- Linux/x86_64 smithi033 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 2024-04-25T01:11:44.219 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:44.573 INFO:teuthology.orchestra.run.smithi033.stdout:generic/099 0s 2024-04-25T01:11:44.579 INFO:teuthology.orchestra.run.smithi033.stdout:Ran: generic/099 2024-04-25T01:11:44.579 INFO:teuthology.orchestra.run.smithi033.stdout:Passed all 1 tests 2024-04-25T01:11:44.582 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:44.584 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[38878]: fuse finished with error 0 and tester_r 0 2024-04-25T01:11:44.593 INFO:tasks.cephfs.test_acls:Command return value: 0 2024-04-25T01:11:44.593 INFO:tasks.cephfs.test_acls:Command stdout - FSTYP -- ceph-fuse PLATFORM -- Linux/x86_64 smithi033 5.15.0-105-generic #115-Ubuntu SMP Mon Apr 15 09:52:04 UTC 2024 generic/099 0s Ran: generic/099 Passed all 1 tests 2024-04-25T01:11:44.593 INFO:tasks.cephfs.test_acls:Command stderr - 2024-04-25T01:11:44.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo userdel --force --remove fsgqa 2024-04-25T01:11:44.667 INFO:teuthology.orchestra.run.smithi033.stderr:userdel: fsgqa mail spool (/var/mail/fsgqa) not found 2024-04-25T01:11:44.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo userdel --force --remove 123456-fsgqa 2024-04-25T01:11:45.055 INFO:teuthology.orchestra.run.smithi033.stderr:userdel: 123456-fsgqa mail spool (/var/mail/123456-fsgqa) not found 2024-04-25T01:11:45.056 INFO:teuthology.orchestra.run.smithi033.stderr:userdel: 123456-fsgqa home directory (/home/123456-fsgqa) not found 2024-04-25T01:11:45.437 DEBUG:teuthology.orchestra.run.smithi033:> sudo groupdel fsgqa 2024-04-25T01:11:45.459 INFO:teuthology.orchestra.run.smithi033.stderr:groupdel: group 'fsgqa' does not exist 2024-04-25T01:11:45.461 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-25T01:11:45.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:45.461 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo stat /tmp/tmp.mSBFkuSi7xxfstests-dev/results) 2024-04-25T01:11:45.508 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: cd: /home/ubuntu/cephtest/mnt.0: Permission denied 2024-04-25T01:11:45.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:11:45.509 INFO:tasks.cephfs.xfstests_dev:xfstests-dev repo contains not directory named "results". repo location: /tmp/tmp.mSBFkuSi7xxfstests-dev 2024-04-25T01:11:45.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf /tmp/tmp.mSBFkuSi7xxfstests-dev 2024-04-25T01:11:45.635 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T01:11:45.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T01:11:45.690 DEBUG:teuthology.orchestra.run.smithi138:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T01:11:45.736 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T01:11:45.736 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T01:11:45.737 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:11:45.737 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:45.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18498 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T01:11:45.743 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:11:45.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:45.743 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T01:11:45.791 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T01:11:45.791 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:11:45.791 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:45.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16945 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T01:11:45.798 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:11:45.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:45.798 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T01:11:45.843 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T01:11:45.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:11:45.844 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T01:11:45.844 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:11:45.844 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:45.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18498 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T01:11:45.850 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:11:45.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:45.850 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T01:11:45.897 INFO:teuthology.orchestra.run.smithi033.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T01:11:45.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:11:45.898 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T01:11:45.898 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:11:45.898 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:45.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16945 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T01:11:45.904 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:11:45.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:45.905 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T01:11:45.951 INFO:teuthology.orchestra.run.smithi138.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T01:11:45.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:11:45.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:11:46.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.095+0000 7f38ce179640 1 Processor -- start 2024-04-25T01:11:46.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38ce179640 1 -- start start 2024-04-25T01:11:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38ce179640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c80b4c50 0x7f38c815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38ce179640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38c80b5680 0x7f38c80b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38ce179640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c806f020 con 0x7f38c81563d0 2024-04-25T01:11:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38ce179640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f38c80b6030 con 0x7f38c80b5680 2024-04-25T01:11:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38ce179640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f38c806f2a0 con 0x7f38c80b4c50 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38c80b5680 0x7f38c80b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f38c80b5680 0x7f38c80b5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36256/0 (socket says 172.21.15.33:36256) 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 -- 172.21.15.33:0/1947744108 learned_addr learned my addr 172.21.15.33:0/1947744108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c6ffd640 1 --2- 172.21.15.33:0/1947744108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c80b4c50 0x7f38c815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 -- 172.21.15.33:0/1947744108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c80b4c50 msgr2=0x7f38c815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 --2- 172.21.15.33:0/1947744108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c80b4c50 0x7f38c815d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 -- 172.21.15.33:0/1947744108 >> v1:172.21.15.138:6789/0 conn(0x7f38c81563d0 legacy=0x7f38c815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 -- 172.21.15.33:0/1947744108 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c806f580 con 0x7f38c80b5680 2024-04-25T01:11:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c6ffd640 1 --2- 172.21.15.33:0/1947744108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c80b4c50 0x7f38c815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.099+0000 7f38c7fff640 1 --2- 172.21.15.33:0/1947744108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b5af0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009870 tx=0x7f38c001c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fea33aa925cc0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c67fc640 1 -- 172.21.15.33:0/1947744108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00b0070 con 0x7f38c80b5680 2024-04-25T01:11:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c67fc640 1 -- 172.21.15.33:0/1947744108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c00ab600 con 0x7f38c80b5680 2024-04-25T01:11:46.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c67fc640 1 -- 172.21.15.33:0/1947744108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00ab9d0 con 0x7f38c80b5680 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- 172.21.15.33:0/1947744108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 msgr2=0x7f38c80b5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 --2- 172.21.15.33:0/1947744108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b5af0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f38c0009870 tx=0x7f38c001c2b0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- 172.21.15.33:0/1947744108 shutdown_connections 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 --2- 172.21.15.33:0/1947744108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b5af0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 --2- 172.21.15.33:0/1947744108 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c80b4c50 0x7f38c815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- 172.21.15.33:0/1947744108 >> 172.21.15.33:0/1947744108 conn(0x7f38c80734c0 msgr2=0x7f38c809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- 172.21.15.33:0/1947744108 shutdown_connections 2024-04-25T01:11:46.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- 172.21.15.33:0/1947744108 wait complete. 2024-04-25T01:11:46.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 Processor -- start 2024-04-25T01:11:46.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- start start 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38c80b4c50 0x7f38c814c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c81563d0 0x7f38c81573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f38c8070ee0 con 0x7f38c80b5680 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38c8070d60 con 0x7f38c80b4c50 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f38c8071060 con 0x7f38c81563d0 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36268/0 (socket says 172.21.15.33:36268) 2024-04-25T01:11:46.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 -- 172.21.15.33:0/2173256963 learned_addr learned my addr 172.21.15.33:0/2173256963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c77fe640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38c80b4c50 0x7f38c814c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c7fff640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c81563d0 0x7f38c81573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 -- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c81563d0 msgr2=0x7f38c81573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c81563d0 0x7f38c81573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 -- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38c80b4c50 msgr2=0x7f38c814c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38c80b4c50 0x7f38c814c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 -- 172.21.15.33:0/2173256963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c814cd30 con 0x7f38c80b5680 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38c6ffd640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b0db0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f38b800ce30 tx=0x7f38b800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38bffff640 1 -- 172.21.15.33:0/2173256963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b800f680 con 0x7f38c80b5680 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38bffff640 1 -- 172.21.15.33:0/2173256963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b800f820 con 0x7f38c80b5680 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38bffff640 1 -- 172.21.15.33:0/2173256963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b800fbf0 con 0x7f38c80b5680 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c81549c0 con 0x7f38c80b5680 2024-04-25T01:11:46.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.103+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c8154f60 con 0x7f38c80b5680 2024-04-25T01:11:46.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.107+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c8069040 con 0x7f38c80b5680 2024-04-25T01:11:46.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.107+0000 7f38bffff640 1 -- 172.21.15.33:0/2173256963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38b8018400 con 0x7f38c80b5680 2024-04-25T01:11:46.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.107+0000 7f38bffff640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38a80678c0 0x7f38a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.107+0000 7f38bffff640 1 -- 172.21.15.33:0/2173256963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f38b8086b20 con 0x7f38c80b5680 2024-04-25T01:11:46.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.107+0000 7f38c77fe640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38a80678c0 0x7f38a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.107+0000 7f38c77fe640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38a80678c0 0x7f38a8069d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f38b0003d60 tx=0x7f38b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.111+0000 7f38bffff640 1 -- 172.21.15.33:0/2173256963 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f38b8014030 con 0x7f38c80b5680 2024-04-25T01:11:46.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.315+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38c80b1840 con 0x7f38c80b5680 2024-04-25T01:11:46.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.315+0000 7f38bffff640 1 -- 172.21.15.33:0/2173256963 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f38b80574b0 con 0x7f38c80b5680 2024-04-25T01:11:46.306 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:46.306 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3228956110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3228956110},{"type":"v1","addr":"172.21.15.138:6835","nonce":3228956110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2116493369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2116493369},{"type":"v1","addr":"172.21.15.33:6835","nonce":2116493369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2610873006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2610873006},{"type":"v1","addr":"172.21.15.138:6837","nonce":2610873006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:09:45.288708+0000","modified":"2024-04-25T01:09:48.318253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/3726672082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3726672082},{"type":"v1","addr":"172.21.15.33:6834","nonce":3726672082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-25T01:11:46.307 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-25T01:11:46.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38a80678c0 msgr2=0x7f38a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38a80678c0 0x7f38a8069d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f38b0003d60 tx=0x7f38b0009040 comp rx=0 tx=0).stop 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 msgr2=0x7f38c80b0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b0db0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f38b800ce30 tx=0x7f38b800a650 comp rx=0 tx=0).stop 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 shutdown_connections 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f38a80678c0 0x7f38a8069d80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f38c81563d0 0x7f38c81573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38c80b5680 0x7f38c80b0db0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 --2- 172.21.15.33:0/2173256963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38c80b4c50 0x7f38c814c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 >> 172.21.15.33:0/2173256963 conn(0x7f38c80734c0 msgr2=0x7f38c814f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 shutdown_connections 2024-04-25T01:11:46.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.319+0000 7f38ce179640 1 -- 172.21.15.33:0/2173256963 wait complete. 2024-04-25T01:11:46.327 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:11:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f0101cca640 1 Processor -- start 2024-04-25T01:11:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f0101cca640 1 -- start start 2024-04-25T01:11:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f0101cca640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f00fc0b5530 0x7f00fc166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f0101cca640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00fc15f9a0 con 0x7f00fc145470 2024-04-25T01:11:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f0101cca640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00fc15fc80 con 0x7f00fc15c160 2024-04-25T01:11:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f0101cca640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f00fc15f750 con 0x7f00fc0b5530 2024-04-25T01:11:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fb7fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f00fc145470 0x7f00fc0cf0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44324/0 (socket says 172.21.15.33:44324) 2024-04-25T01:11:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fb7fe640 1 -- 172.21.15.33:0/2834365145 learned_addr learned my addr 172.21.15.33:0/2834365145 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 --2- 172.21.15.33:0/2834365145 >> v2:172.21.15.138:3300/0 conn(0x7f00fc0b5530 0x7f00fc166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2703096211 0 0) 0x7f00fc15f9a0 con 0x7f00fc145470 2024-04-25T01:11:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00e4003530 con 0x7f00fc145470 2024-04-25T01:11:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1054941641 0 0) 0x7f00e4003530 con 0x7f00fc145470 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 >> v2:172.21.15.138:3300/0 conn(0x7f00fc0b5530 msgr2=0x7f00fc166ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 --2- 172.21.15.33:0/2834365145 >> v2:172.21.15.138:3300/0 conn(0x7f00fc0b5530 0x7f00fc166ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 >> v1:172.21.15.138:6789/0 conn(0x7f00fc15c160 legacy=0x7f00fc0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc15ff60 con 0x7f00fc145470 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f00ec001680 con 0x7f00fc145470 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 --2- 172.21.15.33:0/2834365145 >> v2:172.21.15.138:3300/0 conn(0x7f00fc0b5530 0x7f00fc166ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 >> v1:172.21.15.33:6789/0 conn(0x7f00fc145470 legacy=0x7f00fc0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00e40039c0 0x7f00e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e4004810 0x7f00e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e40051d0 0x7f00e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc15f750 con 0x7f00e40051d0 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00e400c120 con 0x7f00e40039c0 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00e400c320 con 0x7f00e4004810 2024-04-25T01:11:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e40051d0 0x7f00e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fb7fe640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00e40039c0 0x7f00e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00faffd640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e4004810 0x7f00e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 -- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e4004810 msgr2=0x7f00e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e4004810 0x7f00e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 -- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00e40039c0 msgr2=0x7f00e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00e40039c0 0x7f00e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.455+0000 7f00fbfff640 1 -- 172.21.15.33:0/2834365145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e4003530 con 0x7f00e40051d0 2024-04-25T01:11:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f00fbfff640 1 -- 172.21.15.33:0/2834365145 reap_dead start 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f00fbfff640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e40051d0 0x7f00e400b9d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f00f40073e0 tx=0x7f00f401cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2aa77858096da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4009c80 con 0x7f00e40051d0 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f40ad440 con 0x7f00e40051d0 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f00fa7fc640 1 -- 172.21.15.33:0/2834365145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f40ad830 con 0x7f00e40051d0 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 -- 172.21.15.33:0/2834365145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e40051d0 msgr2=0x7f00e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e40051d0 0x7f00e400b9d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f00f40073e0 tx=0x7f00f401cf40 comp rx=0 tx=0).stop 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 -- 172.21.15.33:0/2834365145 shutdown_connections 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 --2- 172.21.15.33:0/2834365145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00e40051d0 0x7f00e400b9d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 -- 172.21.15.33:0/2834365145 >> 172.21.15.33:0/2834365145 conn(0x7f00fc0b1810 msgr2=0x7f00fc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 -- 172.21.15.33:0/2834365145 shutdown_connections 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 -- 172.21.15.33:0/2834365145 wait complete. 2024-04-25T01:11:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 Processor -- start 2024-04-25T01:11:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.459+0000 7f0101cca640 1 -- start start 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e40051d0 0x7f00fc1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc13b430 0x7f00fc15dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00fc13f050 0x7f00fc13f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc06e850 con 0x7f00fc13b430 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc145850 con 0x7f00fc13f050 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc0b4e20 con 0x7f00e40051d0 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc13b430 0x7f00fc15dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc13b430 0x7f00fc15dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36286/0 (socket says 172.21.15.33:36286) 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 -- 172.21.15.33:0/3424627915 learned_addr learned my addr 172.21.15.33:0/3424627915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00fbfff640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00fc13f050 0x7f00fc13f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00fb7fe640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e40051d0 0x7f00fc1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 -- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e40051d0 msgr2=0x7f00fc1f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e40051d0 0x7f00fc1f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 -- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00fc13f050 msgr2=0x7f00fc13f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00fc13f050 0x7f00fc13f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 -- 172.21.15.33:0/3424627915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc1f4640 con 0x7f00fc13b430 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00fbfff640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00fc13f050 0x7f00fc13f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00faffd640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc13b430 0x7f00fc15dec0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f00f000e4e0 tx=0x7f00f000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00e3fff640 1 -- 172.21.15.33:0/3424627915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f00052e0 con 0x7f00fc13b430 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00e3fff640 1 -- 172.21.15.33:0/3424627915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f0018070 con 0x7f00fc13b430 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc13b900 con 0x7f00fc13b430 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00e3fff640 1 -- 172.21.15.33:0/3424627915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0005480 con 0x7f00fc13b430 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc13bec0 con 0x7f00fc13b430 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc069040 con 0x7f00fc13b430 2024-04-25T01:11:46.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00e3fff640 1 -- 172.21.15.33:0/3424627915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00f0003840 con 0x7f00fc13b430 2024-04-25T01:11:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00e3fff640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00e80678c0 0x7f00e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00e3fff640 1 -- 172.21.15.33:0/3424627915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f00f0085a80 con 0x7f00fc13b430 2024-04-25T01:11:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.463+0000 7f00fb7fe640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00e80678c0 0x7f00e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.467+0000 7f00fb7fe640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00e80678c0 0x7f00e8069d80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f00fc15db90 tx=0x7f00ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.467+0000 7f00e3fff640 1 -- 172.21.15.33:0/3424627915 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f00f0056410 con 0x7f00fc13b430 2024-04-25T01:11:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.631+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00fc071420 con 0x7f00fc13b430 2024-04-25T01:11:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f00e3fff640 1 -- 172.21.15.33:0/3424627915 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16923 (secure 0 0 0) 0x7f00f0055d10 con 0x7f00fc13b430 2024-04-25T01:11:46.626 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:46.626 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":41,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:10:13.216905+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:09:43.725950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T01:09:44.271067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:11:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00e80678c0 msgr2=0x7f00e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00e80678c0 0x7f00e8069d80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f00fc15db90 tx=0x7f00ec009040 comp rx=0 tx=0).stop 2024-04-25T01:11:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc13b430 msgr2=0x7f00fc15dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc13b430 0x7f00fc15dec0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f00f000e4e0 tx=0x7f00f000e9a0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 shutdown_connections 2024-04-25T01:11:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f00e80678c0 0x7f00e8069d80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00fc13f050 0x7f00fc13f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc13b430 0x7f00fc15dec0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 --2- 172.21.15.33:0/3424627915 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f00e40051d0 0x7f00fc1f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 >> 172.21.15.33:0/3424627915 conn(0x7f00fc0b1810 msgr2=0x7f00fc1446d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.635+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 shutdown_connections 2024-04-25T01:11:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.639+0000 7f0101cca640 1 -- 172.21.15.33:0/3424627915 wait complete. 2024-04-25T01:11:46.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:08:15.214297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T01:09:43.725950+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T01:09:44.271067+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T01:11:46.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:11:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.783+0000 7f91a17b8640 1 Processor -- start 2024-04-25T01:11:46.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.783+0000 7f91a17b8640 1 -- start start 2024-04-25T01:11:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f91a17b8640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f919c15ae20 0x7f919c0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f91a17b8640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f919c072410 0x7f919c0b4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f91a17b8640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c0b4a20 con 0x7f919c06e5b0 2024-04-25T01:11:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f91a17b8640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f919c0b2bb0 con 0x7f919c072410 2024-04-25T01:11:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f91a17b8640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f919c0b2e90 con 0x7f919c15ae20 2024-04-25T01:11:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919a7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f919c072410 0x7f919c0b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919a7fc640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f919c072410 0x7f919c0b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36298/0 (socket says 172.21.15.33:36298) 2024-04-25T01:11:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919a7fc640 1 -- 172.21.15.33:0/2943348973 learned_addr learned my addr 172.21.15.33:0/2943348973 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919affd640 1 --2- 172.21.15.33:0/2943348973 >> v2:172.21.15.138:3300/0 conn(0x7f919c15ae20 0x7f919c0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919a7fc640 1 -- 172.21.15.33:0/2943348973 >> v2:172.21.15.138:3300/0 conn(0x7f919c15ae20 msgr2=0x7f919c0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919a7fc640 1 --2- 172.21.15.33:0/2943348973 >> v2:172.21.15.138:3300/0 conn(0x7f919c15ae20 0x7f919c0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919a7fc640 1 -- 172.21.15.33:0/2943348973 >> v1:172.21.15.138:6789/0 conn(0x7f919c06e5b0 legacy=0x7f919c15e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:46.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.787+0000 7f919a7fc640 1 -- 172.21.15.33:0/2943348973 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c0b3170 con 0x7f919c072410 2024-04-25T01:11:46.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f919a7fc640 1 --2- 172.21.15.33:0/2943348973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c072410 0x7f919c0b4340 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f919c06c190 tx=0x7f919001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af2e3fff4e33b7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f9199ffb640 1 -- 172.21.15.33:0/2943348973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91900aa660 con 0x7f919c072410 2024-04-25T01:11:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f9199ffb640 1 -- 172.21.15.33:0/2943348973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91900af070 con 0x7f919c072410 2024-04-25T01:11:46.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f9199ffb640 1 -- 172.21.15.33:0/2943348973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91900aaa20 con 0x7f919c072410 2024-04-25T01:11:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- 172.21.15.33:0/2943348973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c072410 msgr2=0x7f919c0b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2943348973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c072410 0x7f919c0b4340 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f919c06c190 tx=0x7f919001c4e0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- 172.21.15.33:0/2943348973 shutdown_connections 2024-04-25T01:11:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2943348973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c072410 0x7f919c0b4340 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2943348973 >> v2:172.21.15.138:3300/0 conn(0x7f919c15ae20 0x7f919c0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- 172.21.15.33:0/2943348973 >> 172.21.15.33:0/2943348973 conn(0x7f919c0b1810 msgr2=0x7f919c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:46.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- 172.21.15.33:0/2943348973 shutdown_connections 2024-04-25T01:11:46.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- 172.21.15.33:0/2943348973 wait complete. 2024-04-25T01:11:46.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 Processor -- start 2024-04-25T01:11:46.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- start start 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c06e5b0 0x7f919c15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919c072410 0x7f919c13a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919c15ae20 0x7f919c15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f919c165d70 con 0x7f919c06e5b0 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f919c165bf0 con 0x7f919c072410 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f91a17b8640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f919c165ef0 con 0x7f919c15ae20 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f919affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c06e5b0 0x7f919c15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f919affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c06e5b0 0x7f919c15d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36310/0 (socket says 172.21.15.33:36310) 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.791+0000 7f919affd640 1 -- 172.21.15.33:0/2393193679 learned_addr learned my addr 172.21.15.33:0/2393193679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919a7fc640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919c072410 0x7f919c13a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919b7fe640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919c15ae20 0x7f919c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919affd640 1 -- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919c15ae20 msgr2=0x7f919c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919affd640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919c15ae20 0x7f919c15a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919affd640 1 -- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919c072410 msgr2=0x7f919c13a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919affd640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919c072410 0x7f919c13a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919affd640 1 -- 172.21.15.33:0/2393193679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c15a670 con 0x7f919c06e5b0 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919affd640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c06e5b0 0x7f919c15d9f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f918c00d620 tx=0x7f918c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91837fe640 1 -- 172.21.15.33:0/2393193679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00b620 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91837fe640 1 -- 172.21.15.33:0/2393193679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918c019070 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c156660 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91837fe640 1 -- 172.21.15.33:0/2393193679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c00ba00 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c15bdb0 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c069040 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91837fe640 1 -- 172.21.15.33:0/2393193679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f918c00bba0 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91837fe640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91880678c0 0x7f9188069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f91837fe640 1 -- 172.21.15.33:0/2393193679 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f918c086280 con 0x7f919c06e5b0 2024-04-25T01:11:46.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919a7fc640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91880678c0 0x7f9188069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.795+0000 7f919a7fc640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91880678c0 0x7f9188069d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f91900ae000 tx=0x7f91900a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:46.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.799+0000 7f91837fe640 1 -- 172.21.15.33:0/2393193679 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f918c056bf0 con 0x7f919c06e5b0 2024-04-25T01:11:46.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.955+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f919c06ce50 con 0x7f919c06e5b0 2024-04-25T01:11:46.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.955+0000 7f91837fe640 1 -- 172.21.15.33:0/2393193679 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f918c0564f0 con 0x7f919c06e5b0 2024-04-25T01:11:46.947 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:11:46.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91880678c0 msgr2=0x7f9188069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91880678c0 0x7f9188069d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f91900ae000 tx=0x7f91900a7000 comp rx=0 tx=0).stop 2024-04-25T01:11:46.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c06e5b0 msgr2=0x7f919c15d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:46.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c06e5b0 0x7f919c15d9f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f918c00d620 tx=0x7f918c00dae0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 shutdown_connections 2024-04-25T01:11:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f91880678c0 0x7f9188069d80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f919c15ae20 0x7f919c15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f919c072410 0x7f919c13a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 --2- 172.21.15.33:0/2393193679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f919c06e5b0 0x7f919c15d9f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 >> 172.21.15.33:0/2393193679 conn(0x7f919c0b1810 msgr2=0x7f919c157c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 shutdown_connections 2024-04-25T01:11:46.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:46.959+0000 7f91a17b8640 1 -- 172.21.15.33:0/2393193679 wait complete. 2024-04-25T01:11:46.965 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T01:11:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06e089e640 1 Processor -- start 2024-04-25T01:11:47.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06e089e640 1 -- start start 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06e089e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06dc0cf460 0x7f06dc0d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06e089e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f06dc06c7a0 0x7f06dc06cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06e089e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06dc06bd00 con 0x7f06dc0d1ec0 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06e089e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f06dc0cf920 con 0x7f06dc0cf460 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06e089e640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f06dc0cfc00 con 0x7f06dc06c7a0 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06dc0cf460 0x7f06dc0d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06dc0cf460 0x7f06dc0d4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36314/0 (socket says 172.21.15.33:36314) 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3248624324 learned_addr learned my addr 172.21.15.33:0/3248624324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06da59c640 1 --1- 172.21.15.33:0/3248624324 >> v1:172.21.15.33:6789/0 conn(0x7f06dc0d1ec0 0x7f06dc0d22a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44334/0 (socket says 172.21.15.33:44334) 2024-04-25T01:11:47.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06dad9d640 1 --2- 172.21.15.33:0/3248624324 >> v2:172.21.15.138:3300/0 conn(0x7f06dc06c7a0 0x7f06dc06cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3248624324 >> v2:172.21.15.138:3300/0 conn(0x7f06dc06c7a0 msgr2=0x7f06dc06cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 --2- 172.21.15.33:0/3248624324 >> v2:172.21.15.138:3300/0 conn(0x7f06dc06c7a0 0x7f06dc06cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3248624324 >> v1:172.21.15.33:6789/0 conn(0x7f06dc0d1ec0 legacy=0x7f06dc0d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3248624324 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc06bfa0 con 0x7f06dc0cf460 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d1d9b640 1 --2- 172.21.15.33:0/3248624324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc0d4960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009870 tx=0x7f06cc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd780c39b2f8064e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d9d9b640 1 -- 172.21.15.33:0/3248624324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0aa670 con 0x7f06dc0cf460 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.099+0000 7f06d9d9b640 1 -- 172.21.15.33:0/3248624324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc0af070 con 0x7f06dc0cf460 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d9d9b640 1 -- 172.21.15.33:0/3248624324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0aaa30 con 0x7f06dc0cf460 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- 172.21.15.33:0/3248624324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 msgr2=0x7f06dc0d4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 --2- 172.21.15.33:0/3248624324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc0d4960 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f06cc009870 tx=0x7f06cc01c4f0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- 172.21.15.33:0/3248624324 shutdown_connections 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 --2- 172.21.15.33:0/3248624324 >> v2:172.21.15.138:3300/0 conn(0x7f06dc06c7a0 0x7f06dc06cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 --2- 172.21.15.33:0/3248624324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc0d4960 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- 172.21.15.33:0/3248624324 >> 172.21.15.33:0/3248624324 conn(0x7f06dc0d3e40 msgr2=0x7f06dc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:47.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- 172.21.15.33:0/3248624324 shutdown_connections 2024-04-25T01:11:47.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- 172.21.15.33:0/3248624324 wait complete. 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 Processor -- start 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- start start 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06dc06c7a0 0x7f06dc0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06dc0d1ec0 0x7f06dc0b8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc16ac70 con 0x7f06dc0cf460 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc16aaf0 con 0x7f06dc0d1ec0 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06dc16adf0 con 0x7f06dc06c7a0 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36320/0 (socket says 172.21.15.33:36320) 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3206214736 learned_addr learned my addr 172.21.15.33:0/3206214736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06da59c640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06dc06c7a0 0x7f06dc0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06dad9d640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06dc0d1ec0 0x7f06dc0b8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06dc06c7a0 msgr2=0x7f06dc0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06dc06c7a0 0x7f06dc0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06dc0d1ec0 msgr2=0x7f06dc0b8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06dc0d1ec0 0x7f06dc0b8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 -- 172.21.15.33:0/3206214736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc0b73f0 con 0x7f06dc0cf460 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d1d9b640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc154000 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06cc01a120 tx=0x7f06cc01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d37fe640 1 -- 172.21.15.33:0/3206214736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0bd5f0 con 0x7f06dc0cf460 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d37fe640 1 -- 172.21.15.33:0/3206214736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06cc0bd790 con 0x7f06dc0cf460 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06d37fe640 1 -- 172.21.15.33:0/3206214736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06cc0bdb80 con 0x7f06dc0cf460 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc0ca650 con 0x7f06dc0cf460 2024-04-25T01:11:47.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.103+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc0b7970 con 0x7f06dc0cf460 2024-04-25T01:11:47.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.107+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06dc069040 con 0x7f06dc0cf460 2024-04-25T01:11:47.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.107+0000 7f06d37fe640 1 -- 172.21.15.33:0/3206214736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06dc06d980 con 0x7f06dc0cf460 2024-04-25T01:11:47.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.107+0000 7f06d37fe640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.107+0000 7f06d37fe640 1 -- 172.21.15.33:0/3206214736 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f06cc120930 con 0x7f06dc0cf460 2024-04-25T01:11:47.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.107+0000 7f06da59c640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.107+0000 7f06da59c640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0ce210 tx=0x7f06c4007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.111+0000 7f06d37fe640 1 -- 172.21.15.33:0/3206214736 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f06cc016110 con 0x7f06dc0cf460 2024-04-25T01:11:47.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f06dc069040 con 0x7f06dc0cf460 2024-04-25T01:11:47.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06d37fe640 1 -- 172.21.15.33:0/3206214736 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) ==== 104+0+11 (secure 0 0 0) 0x7f06cc0f1350 con 0x7f06dc0cf460 2024-04-25T01:11:47.262 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 msgr2=0x7f06bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0ce210 tx=0x7f06c4007cf0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 msgr2=0x7f06dc154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc154000 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f06cc01a120 tx=0x7f06cc01ad30 comp rx=0 tx=0).stop 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 shutdown_connections 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06dc0d1ec0 0x7f06dc0b8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06dc0cf460 0x7f06dc154000 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 --2- 172.21.15.33:0/3206214736 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06dc06c7a0 0x7f06dc0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 >> 172.21.15.33:0/3206214736 conn(0x7f06dc0d3e40 msgr2=0x7f06dc0b41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 shutdown_connections 2024-04-25T01:11:47.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.271+0000 7f06e089e640 1 -- 172.21.15.33:0/3206214736 wait complete. 2024-04-25T01:11:47.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T01:11:47.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 Processor -- start 2024-04-25T01:11:47.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 -- start start 2024-04-25T01:11:47.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb380720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcb381587b0 0x7fcb3806c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb38072630 0x7fcb3806c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fcb380b4c10 con 0x7fcb38072630 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fcb380b4000 con 0x7fcb381587b0 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3ef95640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcb380b42e0 con 0x7fcb381583d0 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb38072630 0x7fcb3806c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fcb38072630 0x7fcb3806c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36330/0 (socket says 172.21.15.33:36330) 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 -- 172.21.15.33:0/2864829104 learned_addr learned my addr 172.21.15.33:0/2864829104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb37fff640 1 --2- 172.21.15.33:0/2864829104 >> v2:172.21.15.138:3300/0 conn(0x7fcb381587b0 0x7fcb3806c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3cd31640 1 --2- 172.21.15.33:0/2864829104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb380720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 -- 172.21.15.33:0/2864829104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 msgr2=0x7fcb380720f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 --2- 172.21.15.33:0/2864829104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb380720f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 -- 172.21.15.33:0/2864829104 >> v2:172.21.15.138:3300/0 conn(0x7fcb381587b0 msgr2=0x7fcb3806c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 --2- 172.21.15.33:0/2864829104 >> v2:172.21.15.138:3300/0 conn(0x7fcb381587b0 0x7fcb3806c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.419+0000 7fcb3d532640 1 -- 172.21.15.33:0/2864829104 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb380b45c0 con 0x7fcb38072630 2024-04-25T01:11:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3d532640 1 --2- 172.21.15.33:0/2864829104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3806c760 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcb38072ef0 tx=0x7fcb3001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce24668436fe9218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb377fe640 1 -- 172.21.15.33:0/2864829104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb300a5860 con 0x7fcb38072630 2024-04-25T01:11:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb377fe640 1 -- 172.21.15.33:0/2864829104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb300a5a00 con 0x7fcb38072630 2024-04-25T01:11:47.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb377fe640 1 -- 172.21.15.33:0/2864829104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb300a5dd0 con 0x7fcb38072630 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- 172.21.15.33:0/2864829104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 msgr2=0x7fcb3806c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/2864829104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3806c760 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fcb38072ef0 tx=0x7fcb3001c9c0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- 172.21.15.33:0/2864829104 shutdown_connections 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/2864829104 >> v2:172.21.15.138:3300/0 conn(0x7fcb381587b0 0x7fcb3806c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/2864829104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3806c760 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/2864829104 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb380720f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- 172.21.15.33:0/2864829104 >> 172.21.15.33:0/2864829104 conn(0x7fcb380715d0 msgr2=0x7fcb380719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- 172.21.15.33:0/2864829104 shutdown_connections 2024-04-25T01:11:47.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- 172.21.15.33:0/2864829104 wait complete. 2024-04-25T01:11:47.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 Processor -- start 2024-04-25T01:11:47.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- start start 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb3814b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb381587b0 0x7fcb3815b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb38162660 con 0x7fcb38072630 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcb381624e0 con 0x7fcb381587b0 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3ef95640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fcb381627e0 con 0x7fcb381583d0 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3cd31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3cd31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3814aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36342/0 (socket says 172.21.15.33:36342) 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3cd31640 1 -- 172.21.15.33:0/4160360249 learned_addr learned my addr 172.21.15.33:0/4160360249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb3d532640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb381587b0 0x7fcb3815b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.423+0000 7fcb37fff640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb3814b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3cd31640 1 -- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 msgr2=0x7fcb3814b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3cd31640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb3814b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3cd31640 1 -- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb381587b0 msgr2=0x7fcb3815b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3cd31640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb381587b0 0x7fcb3815b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3cd31640 1 -- 172.21.15.33:0/4160360249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3815a100 con 0x7fcb38072630 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3cd31640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3814aae0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800d5f0 tx=0x7fcb2800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb357fa640 1 -- 172.21.15.33:0/4160360249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb280125f0 con 0x7fcb38072630 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb357fa640 1 -- 172.21.15.33:0/4160360249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb28019070 con 0x7fcb38072630 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3815a3e0 con 0x7fcb38072630 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb357fa640 1 -- 172.21.15.33:0/4160360249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb280129b0 con 0x7fcb38072630 2024-04-25T01:11:47.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3815a9a0 con 0x7fcb38072630 2024-04-25T01:11:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb3814f330 con 0x7fcb38072630 2024-04-25T01:11:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb357fa640 1 -- 172.21.15.33:0/4160360249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcb28012b50 con 0x7fcb38072630 2024-04-25T01:11:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb357fa640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb240678c0 0x7fcb24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb357fa640 1 -- 172.21.15.33:0/4160360249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fcb28086280 con 0x7fcb38072630 2024-04-25T01:11:47.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb37fff640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb240678c0 0x7fcb24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.427+0000 7fcb37fff640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb240678c0 0x7fcb24069d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb380cfcf0 tx=0x7fcb2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.431+0000 7fcb357fa640 1 -- 172.21.15.33:0/4160360249 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fcb3814f330 con 0x7fcb38072630 2024-04-25T01:11:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.591+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcb38069040 con 0x7fcb38072630 2024-04-25T01:11:47.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.591+0000 7fcb357fa640 1 -- 172.21.15.33:0/4160360249 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) ==== 100+0+11 (secure 0 0 0) 0x7fcb28056c10 con 0x7fcb38072630 2024-04-25T01:11:47.584 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb240678c0 msgr2=0x7fcb24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb240678c0 0x7fcb24069d80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcb380cfcf0 tx=0x7fcb2c008040 comp rx=0 tx=0).stop 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 msgr2=0x7fcb3814aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3814aae0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800d5f0 tx=0x7fcb2800dab0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 shutdown_connections 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fcb240678c0 0x7fcb24069d80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb381587b0 0x7fcb3815b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fcb381583d0 0x7fcb3814b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 --2- 172.21.15.33:0/4160360249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb38072630 0x7fcb3814aae0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 >> 172.21.15.33:0/4160360249 conn(0x7fcb380715d0 msgr2=0x7fcb3806d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 shutdown_connections 2024-04-25T01:11:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.595+0000 7fcb3ef95640 1 -- 172.21.15.33:0/4160360249 wait complete. 2024-04-25T01:11:47.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:11:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.723+0000 7f67c884d640 1 Processor -- start 2024-04-25T01:11:47.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.723+0000 7f67c884d640 1 -- start start 2024-04-25T01:11:47.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f67c406ec60 0x7f67c406c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c406e000 con 0x7f67c406f040 2024-04-25T01:11:47.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f67c406e2e0 con 0x7f67c406ec60 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f67c406e5c0 con 0x7f67c40b4b30 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c259c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f67c406ec60 0x7f67c406c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c1d9b640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f67c406f040 0x7f67c415ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44344/0 (socket says 172.21.15.33:44344) 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c1d9b640 1 -- 172.21.15.33:0/3911294205 learned_addr learned my addr 172.21.15.33:0/3911294205 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c2d9d640 1 --2- 172.21.15.33:0/3911294205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c259c640 1 -- 172.21.15.33:0/3911294205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 msgr2=0x7f67c415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c259c640 1 --2- 172.21.15.33:0/3911294205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c259c640 1 -- 172.21.15.33:0/3911294205 >> v1:172.21.15.33:6789/0 conn(0x7f67c406f040 legacy=0x7f67c415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c259c640 1 -- 172.21.15.33:0/3911294205 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c406e8a0 con 0x7f67c406ec60 2024-04-25T01:11:47.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c2d9d640 1 --2- 172.21.15.33:0/3911294205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:47.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c259c640 1 --2- 172.21.15.33:0/3911294205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c406c4b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f67b400ba60 tx=0x7f67b401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeffce3afc7c8ec0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c159a640 1 -- 172.21.15.33:0/3911294205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40b4070 con 0x7f67c406ec60 2024-04-25T01:11:47.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c159a640 1 -- 172.21.15.33:0/3911294205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b40a7a30 con 0x7f67c406ec60 2024-04-25T01:11:47.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c159a640 1 -- 172.21.15.33:0/3911294205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40a7e00 con 0x7f67c406ec60 2024-04-25T01:11:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- 172.21.15.33:0/3911294205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 msgr2=0x7f67c406c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 --2- 172.21.15.33:0/3911294205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c406c4b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f67b400ba60 tx=0x7f67b401eb70 comp rx=0 tx=0).stop 2024-04-25T01:11:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- 172.21.15.33:0/3911294205 shutdown_connections 2024-04-25T01:11:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 --2- 172.21.15.33:0/3911294205 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 --2- 172.21.15.33:0/3911294205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c406c4b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- 172.21.15.33:0/3911294205 >> 172.21.15.33:0/3911294205 conn(0x7f67c40d0270 msgr2=0x7f67c40d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:47.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- 172.21.15.33:0/3911294205 shutdown_connections 2024-04-25T01:11:47.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.727+0000 7f67c884d640 1 -- 172.21.15.33:0/3911294205 wait complete. 2024-04-25T01:11:47.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 Processor -- start 2024-04-25T01:11:47.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 -- start start 2024-04-25T01:11:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c414a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c406f040 0x7f67c414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f67c415b240 con 0x7f67c406ec60 2024-04-25T01:11:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67c415b0c0 con 0x7f67c406f040 2024-04-25T01:11:47.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f67c40afb10 con 0x7f67c40b4b30 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c414a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c414a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36362/0 (socket says 172.21.15.33:36362) 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 -- 172.21.15.33:0/3558236093 learned_addr learned my addr 172.21.15.33:0/3558236093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 -- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 msgr2=0x7f67c415d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 -- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c406f040 msgr2=0x7f67c414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c1d9b640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c406f040 0x7f67c414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c406f040 0x7f67c414abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 -- 172.21.15.33:0/3558236093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41442a0 con 0x7f67c406ec60 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c259c640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c414a680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f67b401c7a0 tx=0x7f67b401b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67b2ffd640 1 -- 172.21.15.33:0/3558236093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40bf070 con 0x7f67c406ec60 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67b2ffd640 1 -- 172.21.15.33:0/3558236093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b40af990 con 0x7f67c406ec60 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41576a0 con 0x7f67c406ec60 2024-04-25T01:11:47.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67b2ffd640 1 -- 172.21.15.33:0/3558236093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40b4040 con 0x7f67c406ec60 2024-04-25T01:11:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c4157c60 con 0x7f67c406ec60 2024-04-25T01:11:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c4157410 con 0x7f67c406ec60 2024-04-25T01:11:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.731+0000 7f67b2ffd640 1 -- 172.21.15.33:0/3558236093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67b40c2070 con 0x7f67c406ec60 2024-04-25T01:11:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.735+0000 7f67b2ffd640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67a40678c0 0x7f67a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.735+0000 7f67b2ffd640 1 -- 172.21.15.33:0/3558236093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f67b411fea0 con 0x7f67c406ec60 2024-04-25T01:11:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.735+0000 7f67c1d9b640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67a40678c0 0x7f67a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.735+0000 7f67c1d9b640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67a40678c0 0x7f67a4069d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cf1a0 tx=0x7f67b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:47.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.739+0000 7f67b2ffd640 1 -- 172.21.15.33:0/3558236093 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f67b4018760 con 0x7f67c406ec60 2024-04-25T01:11:47.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67c4069040 con 0x7f67c406ec60 2024-04-25T01:11:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67b2ffd640 1 -- 172.21.15.33:0/3558236093 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f67b40f08c0 con 0x7f67c406ec60 2024-04-25T01:11:47.917 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:47.918 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3228956110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3228956110},{"type":"v1","addr":"172.21.15.138:6835","nonce":3228956110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2116493369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2116493369},{"type":"v1","addr":"172.21.15.33:6835","nonce":2116493369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2610873006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2610873006},{"type":"v1","addr":"172.21.15.138:6837","nonce":2610873006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:09:45.288708+0000","modified":"2024-04-25T01:09:48.318253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/3726672082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3726672082},{"type":"v1","addr":"172.21.15.33:6834","nonce":3726672082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-25T01:11:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67a40678c0 msgr2=0x7f67a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67a40678c0 0x7f67a4069d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67c40cf1a0 tx=0x7f67b8009040 comp rx=0 tx=0).stop 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 msgr2=0x7f67c414a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c414a680 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f67b401c7a0 tx=0x7f67b401b910 comp rx=0 tx=0).stop 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 shutdown_connections 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67a40678c0 0x7f67a4069d80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f67c40b4b30 0x7f67c415d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67c406f040 0x7f67c414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 --2- 172.21.15.33:0/3558236093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67c406ec60 0x7f67c414a680 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 >> 172.21.15.33:0/3558236093 conn(0x7f67c40d0270 msgr2=0x7f67c406d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 shutdown_connections 2024-04-25T01:11:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:47.927+0000 7f67c884d640 1 -- 172.21.15.33:0/3558236093 wait complete. 2024-04-25T01:11:47.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:11:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfc41e640 1 Processor -- start 2024-04-25T01:11:48.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfc41e640 1 -- start start 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfc41e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 0x7fddf40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfc41e640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf40d10b0 con 0x7fddf40d03e0 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfc41e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddf40d1390 con 0x7fddf415e470 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfc41e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fddf40d1de0 con 0x7fddf40d48d0 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfa1ba640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fddf40d03e0 0x7fddf40d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44348/0 (socket says 172.21.15.33:44348) 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfa1ba640 1 -- 172.21.15.33:0/2127135357 learned_addr learned my addr 172.21.15.33:0/2127135357 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddfa9bb640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 0x7fddf40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 780199631 0 0) 0x7fddf40d10b0 con 0x7fddf40d03e0 2024-04-25T01:11:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.055+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdde0003530 con 0x7fddf40d03e0 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 198519137 0 0) 0x7fdde0003530 con 0x7fddf40d03e0 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 msgr2=0x7fddf40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 0x7fddf40cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 >> v1:172.21.15.138:6789/0 conn(0x7fddf415e470 legacy=0x7fddf40ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf40d1670 con 0x7fddf40d03e0 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fddf0001a00 con 0x7fddf40d03e0 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 >> v1:172.21.15.33:6789/0 conn(0x7fddf40d03e0 legacy=0x7fddf40d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00039f0 0x7fdde0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0004810 0x7fdde0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 0x7fdde0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddf40d1de0 con 0x7fdde00039f0 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdde000bb00 con 0x7fdde0004810 2024-04-25T01:11:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdde000bd00 con 0x7fddf40d48d0 2024-04-25T01:11:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa1ba640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00039f0 0x7fdde0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa9bb640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 0x7fdde0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa1ba640 1 -- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 msgr2=0x7fdde0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa1ba640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf40d48d0 0x7fdde0005210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa1ba640 1 -- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0004810 msgr2=0x7fdde0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa1ba640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdde0004810 0x7fdde0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa1ba640 1 -- 172.21.15.33:0/2127135357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde0003530 con 0x7fdde00039f0 2024-04-25T01:11:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa1ba640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00039f0 0x7fdde0003e10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fddf000a850 tx=0x7fddf001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db374824a7fcb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf000a4a0 con 0x7fdde00039f0 2024-04-25T01:11:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddf00ad650 con 0x7fdde00039f0 2024-04-25T01:11:48.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddf91b8640 1 -- 172.21.15.33:0/2127135357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf00ada40 con 0x7fdde00039f0 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfc41e640 1 -- 172.21.15.33:0/2127135357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00039f0 msgr2=0x7fdde0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfc41e640 1 --2- 172.21.15.33:0/2127135357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdde00039f0 0x7fdde0003e10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fddf000a850 tx=0x7fddf001d3a0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfa9bb640 1 -- 172.21.15.33:0/2127135357 reap_dead start 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfc41e640 1 -- 172.21.15.33:0/2127135357 shutdown_connections 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfc41e640 1 -- 172.21.15.33:0/2127135357 >> 172.21.15.33:0/2127135357 conn(0x7fddf40d3db0 msgr2=0x7fddf40d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfc41e640 1 -- 172.21.15.33:0/2127135357 shutdown_connections 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.059+0000 7fddfc41e640 1 -- 172.21.15.33:0/2127135357 wait complete. 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 Processor -- start 2024-04-25T01:11:48.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 -- start start 2024-04-25T01:11:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf415b770 0x7fddf40c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddf40b85b0 0x7fddf40ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf415bde0 0x7fddf40b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fddf40cfc70 con 0x7fddf40b85b0 2024-04-25T01:11:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddf40cfaf0 con 0x7fddf415bde0 2024-04-25T01:11:48.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fddf40cfdf0 con 0x7fddf415b770 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddf40b85b0 0x7fddf40ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddf40b85b0 0x7fddf40ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36372/0 (socket says 172.21.15.33:36372) 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 -- 172.21.15.33:0/477554861 learned_addr learned my addr 172.21.15.33:0/477554861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfa1ba640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf415b770 0x7fddf40c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfa9bb640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf415bde0 0x7fddf40b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 -- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf415b770 msgr2=0x7fddf40c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf415b770 0x7fddf40c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 -- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf415bde0 msgr2=0x7fddf40b6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf415bde0 0x7fddf40b6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 -- 172.21.15.33:0/477554861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf40b8b20 con 0x7fddf40b85b0 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddf99b9640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddf40b85b0 0x7fddf40ca2c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c500 tx=0x7fdde400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddeaffd640 1 -- 172.21.15.33:0/477554861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40052d0 con 0x7fddf40b85b0 2024-04-25T01:11:48.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddeaffd640 1 -- 172.21.15.33:0/477554861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdde4005470 con 0x7fddf40b85b0 2024-04-25T01:11:48.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddeaffd640 1 -- 172.21.15.33:0/477554861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde4012870 con 0x7fddf40b85b0 2024-04-25T01:11:48.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf409dd50 con 0x7fddf40b85b0 2024-04-25T01:11:48.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf409e310 con 0x7fddf40b85b0 2024-04-25T01:11:48.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf4069040 con 0x7fddf40b85b0 2024-04-25T01:11:48.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.063+0000 7fddeaffd640 1 -- 172.21.15.33:0/477554861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdde4012a10 con 0x7fddf40b85b0 2024-04-25T01:11:48.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.067+0000 7fddeaffd640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fddd80678f0 0x7fddd8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.067+0000 7fddeaffd640 1 -- 172.21.15.33:0/477554861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fdde4085a40 con 0x7fddf40b85b0 2024-04-25T01:11:48.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.067+0000 7fddfa1ba640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fddd80678f0 0x7fddd8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.067+0000 7fddfa1ba640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fddd80678f0 0x7fddd8069db0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fddf00ad000 tx=0x7fddf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.071+0000 7fddeaffd640 1 -- 172.21.15.33:0/477554861 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdde40182b0 con 0x7fddf40b85b0 2024-04-25T01:11:48.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.227+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddf40b5d20 con 0x7fddf40b85b0 2024-04-25T01:11:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddeaffd640 1 -- 172.21.15.33:0/477554861 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16923 (secure 0 0 0) 0x7fdde4056460 con 0x7fddf40b85b0 2024-04-25T01:11:48.221 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:48.221 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":41,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:10:13.216905+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:09:43.725950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T01:09:44.271067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:11:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fddd80678f0 msgr2=0x7fddd8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fddd80678f0 0x7fddd8069db0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fddf00ad000 tx=0x7fddf0009040 comp rx=0 tx=0).stop 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddf40b85b0 msgr2=0x7fddf40ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddf40b85b0 0x7fddf40ca2c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdde400c500 tx=0x7fdde400c9c0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 shutdown_connections 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fddd80678f0 0x7fddd8069db0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddf415bde0 0x7fddf40b6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddf40b85b0 0x7fddf40ca2c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 --2- 172.21.15.33:0/477554861 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fddf415b770 0x7fddf40c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 >> 172.21.15.33:0/477554861 conn(0x7fddf40d3db0 msgr2=0x7fddf415e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 shutdown_connections 2024-04-25T01:11:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.231+0000 7fddfc41e640 1 -- 172.21.15.33:0/477554861 wait complete. 2024-04-25T01:11:48.240 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T01:11:48.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:11:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b7121640 1 Processor -- start 2024-04-25T01:11:48.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b7121640 1 -- start start 2024-04-25T01:11:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b7121640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15b015c160 0x7f15b00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b7121640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b015f9a0 con 0x7f15b00b5360 2024-04-25T01:11:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b7121640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b015fc80 con 0x7f15b0145470 2024-04-25T01:11:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b7121640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f15b00b57b0 con 0x7f15b015c160 2024-04-25T01:11:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b56be640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f15b00b5360 0x7f15b00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:44364/0 (socket says 172.21.15.33:44364) 2024-04-25T01:11:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15affff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f15b015c160 0x7f15b00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15b56be640 1 -- 172.21.15.33:0/3200969684 learned_addr learned my addr 172.21.15.33:0/3200969684 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15affff640 1 -- 172.21.15.33:0/3200969684 >> v1:172.21.15.138:6789/0 conn(0x7f15b0145470 legacy=0x7f15b00cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15affff640 1 -- 172.21.15.33:0/3200969684 >> v1:172.21.15.33:6789/0 conn(0x7f15b00b5360 legacy=0x7f15b00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15affff640 1 -- 172.21.15.33:0/3200969684 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b015ff60 con 0x7f15b015c160 2024-04-25T01:11:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15affff640 1 --2- 172.21.15.33:0/3200969684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b00b4e20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097c0 tx=0x7f159c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949028c0b688eeb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.367+0000 7f15af7fe640 1 -- 172.21.15.33:0/3200969684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c0b0070 con 0x7f15b015c160 2024-04-25T01:11:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15af7fe640 1 -- 172.21.15.33:0/3200969684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f159c0ab600 con 0x7f15b015c160 2024-04-25T01:11:48.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15af7fe640 1 -- 172.21.15.33:0/3200969684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c0ab9d0 con 0x7f15b015c160 2024-04-25T01:11:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- 172.21.15.33:0/3200969684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 msgr2=0x7f15b00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 --2- 172.21.15.33:0/3200969684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b00b4e20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f159c0097c0 tx=0x7f159c01c460 comp rx=0 tx=0).stop 2024-04-25T01:11:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- 172.21.15.33:0/3200969684 shutdown_connections 2024-04-25T01:11:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 --2- 172.21.15.33:0/3200969684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b00b4e20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- 172.21.15.33:0/3200969684 >> 172.21.15.33:0/3200969684 conn(0x7f15b00b1810 msgr2=0x7f15b00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- 172.21.15.33:0/3200969684 shutdown_connections 2024-04-25T01:11:48.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- 172.21.15.33:0/3200969684 wait complete. 2024-04-25T01:11:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 Processor -- start 2024-04-25T01:11:48.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- start start 2024-04-25T01:11:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15b00b5360 0x7f15b013d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15b0145470 0x7f15b0141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b0141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15b0169b00 con 0x7f15b015c160 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f15b006e850 con 0x7f15b0145470 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f15b0169c80 con 0x7f15b00b5360 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b0141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b0141d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36398/0 (socket says 172.21.15.33:36398) 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 -- 172.21.15.33:0/2322017945 learned_addr learned my addr 172.21.15.33:0/2322017945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15affff640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15b0145470 0x7f15b0141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b4ebd640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15b00b5360 0x7f15b013d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 -- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15b00b5360 msgr2=0x7f15b013d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15b00b5360 0x7f15b013d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 -- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15b0145470 msgr2=0x7f15b0141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15b0145470 0x7f15b0141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 -- 172.21.15.33:0/2322017945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b006e130 con 0x7f15b015c160 2024-04-25T01:11:48.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b56be640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b0141d70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15a800c9f0 tx=0x7f15a800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15ad7fa640 1 -- 172.21.15.33:0/2322017945 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8017070 con 0x7f15b015c160 2024-04-25T01:11:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15ad7fa640 1 -- 172.21.15.33:0/2322017945 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a800ab70 con 0x7f15b015c160 2024-04-25T01:11:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b01424b0 con 0x7f15b015c160 2024-04-25T01:11:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.371+0000 7f15ad7fa640 1 -- 172.21.15.33:0/2322017945 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8012400 con 0x7f15b015c160 2024-04-25T01:11:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.375+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b0142a70 con 0x7f15b015c160 2024-04-25T01:11:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.375+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b0069040 con 0x7f15b015c160 2024-04-25T01:11:48.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.375+0000 7f15ad7fa640 1 -- 172.21.15.33:0/2322017945 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f15a801a070 con 0x7f15b015c160 2024-04-25T01:11:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.375+0000 7f15ad7fa640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f15940678c0 0x7f1594069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.375+0000 7f15ad7fa640 1 -- 172.21.15.33:0/2322017945 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f15a80859f0 con 0x7f15b015c160 2024-04-25T01:11:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.375+0000 7f15b4ebd640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f15940678c0 0x7f1594069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.375+0000 7f15b4ebd640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f15940678c0 0x7f1594069d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b810 tx=0x7f15a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.379+0000 7f15ad7fa640 1 -- 172.21.15.33:0/2322017945 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f15a8055cb0 con 0x7f15b015c160 2024-04-25T01:11:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.575+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15b0143340 con 0x7f15b015c160 2024-04-25T01:11:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15ad7fa640 1 -- 172.21.15.33:0/2322017945 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f15a8059fb0 con 0x7f15b015c160 2024-04-25T01:11:48.569 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:48.569 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3228956110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3228956110},{"type":"v1","addr":"172.21.15.138:6835","nonce":3228956110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2116493369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2116493369},{"type":"v1","addr":"172.21.15.33:6835","nonce":2116493369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2610873006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2610873006},{"type":"v1","addr":"172.21.15.138:6837","nonce":2610873006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:09:45.288708+0000","modified":"2024-04-25T01:09:48.318253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/3726672082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3726672082},{"type":"v1","addr":"172.21.15.33:6834","nonce":3726672082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-25T01:11:48.569 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-25T01:11:48.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f15940678c0 msgr2=0x7f1594069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f15940678c0 0x7f1594069d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b810 tx=0x7f15a4008040 comp rx=0 tx=0).stop 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 msgr2=0x7f15b0141d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b0141d70 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15a800c9f0 tx=0x7f15a800ceb0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 shutdown_connections 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f15940678c0 0x7f1594069d80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15b015c160 0x7f15b0141d70 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15b0145470 0x7f15b0141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 --2- 172.21.15.33:0/2322017945 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f15b00b5360 0x7f15b013d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 >> 172.21.15.33:0/2322017945 conn(0x7f15b00b1810 msgr2=0x7f15b00722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 shutdown_connections 2024-04-25T01:11:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.579+0000 7f15b7121640 1 -- 172.21.15.33:0/2322017945 wait complete. 2024-04-25T01:11:48.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:11:48.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f795aa3e640 1 Processor -- start 2024-04-25T01:11:48.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f795aa3e640 1 -- start start 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f795aa3e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 0x7f79540ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f795aa3e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79540d48d0 0x7f79540cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f795aa3e640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79540d10b0 con 0x7f79540d03e0 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f795aa3e640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f79540d1de0 con 0x7f79540d48d0 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f795aa3e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f79540d1390 con 0x7f795415e470 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f7958fdb640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79540d48d0 0x7f79540cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f7958fdb640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f79540d48d0 0x7f79540cef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42584/0 (socket says 172.21.15.33:42584) 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f7958fdb640 1 -- 172.21.15.33:0/4249457217 learned_addr learned my addr 172.21.15.33:0/4249457217 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f79537fe640 1 --2- 172.21.15.33:0/4249457217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 0x7f79540ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f7958fdb640 1 -- 172.21.15.33:0/4249457217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 msgr2=0x7f79540ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:48.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f7958fdb640 1 --2- 172.21.15.33:0/4249457217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 0x7f79540ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f7958fdb640 1 -- 172.21.15.33:0/4249457217 >> v1:172.21.15.138:6789/0 conn(0x7f79540d03e0 legacy=0x7f79540d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.719+0000 7f7958fdb640 1 -- 172.21.15.33:0/4249457217 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79540d1670 con 0x7f79540d48d0 2024-04-25T01:11:48.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7958fdb640 1 --2- 172.21.15.33:0/4249457217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d48d0 0x7f79540cef00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e50 tx=0x7f794c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60f3c7f945cde4ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7952ffd640 1 -- 172.21.15.33:0/4249457217 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c0a56f0 con 0x7f79540d48d0 2024-04-25T01:11:48.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7952ffd640 1 -- 172.21.15.33:0/4249457217 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f794c0a5890 con 0x7f79540d48d0 2024-04-25T01:11:48.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7952ffd640 1 -- 172.21.15.33:0/4249457217 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f794c0a5c60 con 0x7f79540d48d0 2024-04-25T01:11:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- 172.21.15.33:0/4249457217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d48d0 msgr2=0x7f79540cef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 --2- 172.21.15.33:0/4249457217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d48d0 0x7f79540cef00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f794c009e50 tx=0x7f794c01c960 comp rx=0 tx=0).stop 2024-04-25T01:11:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- 172.21.15.33:0/4249457217 shutdown_connections 2024-04-25T01:11:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 --2- 172.21.15.33:0/4249457217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d48d0 0x7f79540cef00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 --2- 172.21.15.33:0/4249457217 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 0x7f79540ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- 172.21.15.33:0/4249457217 >> 172.21.15.33:0/4249457217 conn(0x7f79540d3db0 msgr2=0x7f79540d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:48.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- 172.21.15.33:0/4249457217 shutdown_connections 2024-04-25T01:11:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- 172.21.15.33:0/4249457217 wait complete. 2024-04-25T01:11:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 Processor -- start 2024-04-25T01:11:48.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- start start 2024-04-25T01:11:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d03e0 0x7f7954151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79540d48d0 0x7f7954156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 0x7f7954157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f795416adc0 con 0x7f79540d03e0 2024-04-25T01:11:48.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79540cfea0 con 0x7f79540d48d0 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f795416af40 con 0x7f795415e470 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d03e0 0x7f7954151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d03e0 0x7f7954151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42590/0 (socket says 172.21.15.33:42590) 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 -- 172.21.15.33:0/586100343 learned_addr learned my addr 172.21.15.33:0/586100343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f79537fe640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79540d48d0 0x7f7954156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 -- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 msgr2=0x7f7954157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 0x7f7954157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 -- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79540d48d0 msgr2=0x7f7954156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79540d48d0 0x7f7954156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 -- 172.21.15.33:0/586100343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7954156630 con 0x7f79540d03e0 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f79537fe640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79540d48d0 0x7f7954156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7953fff640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d03e0 0x7f7954151af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c4b0 tx=0x7f793c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7950ff9640 1 -- 172.21.15.33:0/586100343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c016020 con 0x7f79540d03e0 2024-04-25T01:11:48.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7950ff9640 1 -- 172.21.15.33:0/586100343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f793c012830 con 0x7f79540d03e0 2024-04-25T01:11:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79541596a0 con 0x7f79540d03e0 2024-04-25T01:11:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.723+0000 7f7950ff9640 1 -- 172.21.15.33:0/586100343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c012c20 con 0x7f79540d03e0 2024-04-25T01:11:48.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.727+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79541568a0 con 0x7f79540d03e0 2024-04-25T01:11:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.727+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7954069040 con 0x7f79540d03e0 2024-04-25T01:11:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.727+0000 7f7950ff9640 1 -- 172.21.15.33:0/586100343 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f793c019020 con 0x7f79540d03e0 2024-04-25T01:11:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.727+0000 7f7950ff9640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79340678c0 0x7f7934069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.727+0000 7f7950ff9640 1 -- 172.21.15.33:0/586100343 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f793c085860 con 0x7f79540d03e0 2024-04-25T01:11:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.727+0000 7f79537fe640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79340678c0 0x7f7934069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:48.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.727+0000 7f79537fe640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79340678c0 0x7f7934069d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f79540b94f0 tx=0x7f7944008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:48.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.731+0000 7f7950ff9640 1 -- 172.21.15.33:0/586100343 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f793c020070 con 0x7f79540d03e0 2024-04-25T01:11:48.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f795406ce90 con 0x7f79540d03e0 2024-04-25T01:11:48.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f7950ff9640 1 -- 172.21.15.33:0/586100343 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f793c056150 con 0x7f79540d03e0 2024-04-25T01:11:48.913 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:48.914 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3228956110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3228956110},{"type":"v1","addr":"172.21.15.138:6835","nonce":3228956110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2116493369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2116493369},{"type":"v1","addr":"172.21.15.33:6835","nonce":2116493369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2610873006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2610873006},{"type":"v1","addr":"172.21.15.138:6837","nonce":2610873006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T01:09:45.288708+0000","modified":"2024-04-25T01:09:48.318253+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5520},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5520":{"gid":5520,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.33:6834/3726672082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3726672082},{"type":"v1","addr":"172.21.15.33:6834","nonce":3726672082}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5520,"qdb_cluster":[5520]},"id":2}]} 2024-04-25T01:11:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79340678c0 msgr2=0x7f7934069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79340678c0 0x7f7934069d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f79540b94f0 tx=0x7f7944008040 comp rx=0 tx=0).stop 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d03e0 msgr2=0x7f7954151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d03e0 0x7f7954151af0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f793c00c4b0 tx=0x7f793c00c970 comp rx=0 tx=0).stop 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 shutdown_connections 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f79340678c0 0x7f7934069d80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f795415e470 0x7f7954157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79540d48d0 0x7f7954156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 --2- 172.21.15.33:0/586100343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79540d03e0 0x7f7954151af0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 >> 172.21.15.33:0/586100343 conn(0x7f79540d3db0 msgr2=0x7f7954160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 shutdown_connections 2024-04-25T01:11:48.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:48.923+0000 7f795aa3e640 1 -- 172.21.15.33:0/586100343 wait complete. 2024-04-25T01:11:48.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.051+0000 7fdcc7ee3640 1 Processor -- start 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.051+0000 7fdcc7ee3640 1 -- start start 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdcc0145470 0x7fdcc015c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdcc00cf0e0 0x7fdcc00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc00cf570 0x7fdcc00b5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc015f750 con 0x7fdcc0145470 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc015f9a0 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc015fc80 con 0x7fdcc00cf570 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdcc0145470 0x7fdcc015c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fdcc0145470 0x7fdcc015c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42606/0 (socket says 172.21.15.33:42606) 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3400273731 learned_addr learned my addr 172.21.15.33:0/3400273731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc547e640 1 --2- 172.21.15.33:0/3400273731 >> v2:172.21.15.138:3300/0 conn(0x7fdcc00cf0e0 0x7fdcc00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc6480640 1 --2- 172.21.15.33:0/3400273731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc00cf570 0x7fdcc00b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3400273731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc00cf570 msgr2=0x7fdcc00b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 --2- 172.21.15.33:0/3400273731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc00cf570 0x7fdcc00b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3400273731 >> v2:172.21.15.138:3300/0 conn(0x7fdcc00cf0e0 msgr2=0x7fdcc00b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 --2- 172.21.15.33:0/3400273731 >> v2:172.21.15.138:3300/0 conn(0x7fdcc00cf0e0 0x7fdcc00b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3400273731 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc015ff60 con 0x7fdcc0145470 2024-04-25T01:11:49.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc547e640 1 --2- 172.21.15.33:0/3400273731 >> v2:172.21.15.138:3300/0 conn(0x7fdcc00cf0e0 0x7fdcc00b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc5c7f640 1 --2- 172.21.15.33:0/3400273731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc0145470 0x7fdcc015c160 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00cc3c0 tx=0x7fdcbc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaaabde8348b5950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc4c7d640 1 -- 172.21.15.33:0/3400273731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0aa600 con 0x7fdcc0145470 2024-04-25T01:11:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc4c7d640 1 -- 172.21.15.33:0/3400273731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc0af070 con 0x7fdcc0145470 2024-04-25T01:11:49.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc4c7d640 1 -- 172.21.15.33:0/3400273731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0aa9c0 con 0x7fdcc0145470 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3400273731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc0145470 msgr2=0x7fdcc015c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3400273731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc0145470 0x7fdcc015c160 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00cc3c0 tx=0x7fdcbc01c520 comp rx=0 tx=0).stop 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3400273731 shutdown_connections 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3400273731 >> v2:172.21.15.138:3300/0 conn(0x7fdcc00cf0e0 0x7fdcc00b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3400273731 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc00cf570 0x7fdcc00b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3400273731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc0145470 0x7fdcc015c160 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3400273731 >> 172.21.15.33:0/3400273731 conn(0x7fdcc00b1810 msgr2=0x7fdcc00b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:49.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3400273731 shutdown_connections 2024-04-25T01:11:49.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.055+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3400273731 wait complete. 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 Processor -- start 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 -- start start 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc00cf0e0 0x7fdcc0138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc00cf570 0x7fdcc0159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc0145470 0x7fdcc01590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc0072520 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc00723a0 con 0x7fdcc00cf570 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc00726a0 con 0x7fdcc0145470 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc00cf0e0 0x7fdcc0138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc00cf0e0 0x7fdcc0138380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42612/0 (socket says 172.21.15.33:42612) 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3810334969 learned_addr learned my addr 172.21.15.33:0/3810334969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc547e640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc00cf570 0x7fdcc0159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc6480640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc0145470 0x7fdcc01590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc0145470 msgr2=0x7fdcc01590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc0145470 0x7fdcc01590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc00cf570 msgr2=0x7fdcc0159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc00cf570 0x7fdcc0159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 -- 172.21.15.33:0/3810334969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01388c0 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc5c7f640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc00cf0e0 0x7fdcc0138380 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc01abd0 tx=0x7fdcbc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcb67fc640 1 -- 172.21.15.33:0/3810334969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0bd5f0 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcb67fc640 1 -- 172.21.15.33:0/3810334969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc0bd790 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc013b200 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcb67fc640 1 -- 172.21.15.33:0/3810334969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0bdb80 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc013ba80 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.059+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc0069040 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.063+0000 7fdcb67fc640 1 -- 172.21.15.33:0/3810334969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdcbc0a9070 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.063+0000 7fdcb67fc640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdca40678c0 0x7fdca4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.063+0000 7fdcb67fc640 1 -- 172.21.15.33:0/3810334969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fdcbc120a00 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.063+0000 7fdcc547e640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdca40678c0 0x7fdca4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.063+0000 7fdcc547e640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdca40678c0 0x7fdca4069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcc013d730 tx=0x7fdcb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.067+0000 7fdcb67fc640 1 -- 172.21.15.33:0/3810334969 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fdcbc0c3060 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.223+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc015e6f0 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcb67fc640 1 -- 172.21.15.33:0/3810334969 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16923 (secure 0 0 0) 0x7fdcbc016110 con 0x7fdcc00cf0e0 2024-04-25T01:11:49.217 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:49.217 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":41,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:10:13.216905+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T01:09:43.725950+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T01:09:44.271067+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.39","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdca40678c0 msgr2=0x7fdca4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdca40678c0 0x7fdca4069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdcc013d730 tx=0x7fdcb0008040 comp rx=0 tx=0).stop 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc00cf0e0 msgr2=0x7fdcc0138380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc00cf0e0 0x7fdcc0138380 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc01abd0 tx=0x7fdcbc01ad60 comp rx=0 tx=0).stop 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 shutdown_connections 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fdca40678c0 0x7fdca4069d80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fdcc0145470 0x7fdcc01590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcc00cf570 0x7fdcc0159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 --2- 172.21.15.33:0/3810334969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcc00cf0e0 0x7fdcc0138380 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 >> 172.21.15.33:0/3810334969 conn(0x7fdcc00b1810 msgr2=0x7fdcc015edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:49.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 shutdown_connections 2024-04-25T01:11:49.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.227+0000 7fdcc7ee3640 1 -- 172.21.15.33:0/3810334969 wait complete. 2024-04-25T01:11:49.235 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T01:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06e0294640 1 Processor -- start 2024-04-25T01:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06e0294640 1 -- start start 2024-04-25T01:11:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06e0294640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06d815f680 0x7f06d8066f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06e0294640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06d80cda80 con 0x7f06d80aeed0 2024-04-25T01:11:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06e0294640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06d80b2c70 con 0x7f06d806b210 2024-04-25T01:11:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06e0294640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f06d8067490 con 0x7f06d815f680 2024-04-25T01:11:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06de831640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f06d80aeed0 0x7f06d80cd370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33092/0 (socket says 172.21.15.33:33092) 2024-04-25T01:11:49.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06de831640 1 -- 172.21.15.33:0/2404401627 learned_addr learned my addr 172.21.15.33:0/2404401627 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:49.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06de030640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f06d815f680 0x7f06d8066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06de030640 1 -- 172.21.15.33:0/2404401627 >> v1:172.21.15.138:6789/0 conn(0x7f06d806b210 legacy=0x7f06d80ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:49.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06de030640 1 -- 172.21.15.33:0/2404401627 >> v1:172.21.15.33:6789/0 conn(0x7f06d80aeed0 legacy=0x7f06d80cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.367+0000 7f06de030640 1 -- 172.21.15.33:0/2404401627 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d80ad3e0 con 0x7f06d815f680 2024-04-25T01:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06de030640 1 --2- 172.21.15.33:0/2404401627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d815f680 0x7f06d8066f50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f06d40097c0 tx=0x7f06d401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5447479ad7d925 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06dd02e640 1 -- 172.21.15.33:0/2404401627 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d40b0070 con 0x7f06d815f680 2024-04-25T01:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06dd02e640 1 -- 172.21.15.33:0/2404401627 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06d40ab600 con 0x7f06d815f680 2024-04-25T01:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06dd02e640 1 -- 172.21.15.33:0/2404401627 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d40ab9d0 con 0x7f06d815f680 2024-04-25T01:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 -- 172.21.15.33:0/2404401627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d815f680 msgr2=0x7f06d8066f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 --2- 172.21.15.33:0/2404401627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d815f680 0x7f06d8066f50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f06d40097c0 tx=0x7f06d401c3d0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 -- 172.21.15.33:0/2404401627 shutdown_connections 2024-04-25T01:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 --2- 172.21.15.33:0/2404401627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d815f680 0x7f06d8066f50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 -- 172.21.15.33:0/2404401627 >> 172.21.15.33:0/2404401627 conn(0x7f06d80b50c0 msgr2=0x7f06d80b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 -- 172.21.15.33:0/2404401627 shutdown_connections 2024-04-25T01:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 -- 172.21.15.33:0/2404401627 wait complete. 2024-04-25T01:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.371+0000 7f06e0294640 1 Processor -- start 2024-04-25T01:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 -- start start 2024-04-25T01:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06d806b210 0x7f06d814b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d80aeed0 0x7f06d814df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06d815f680 0x7f06d814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f06d80afb30 con 0x7f06d80aeed0 2024-04-25T01:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06d80af9b0 con 0x7f06d806b210 2024-04-25T01:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f06d816a780 con 0x7f06d815f680 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d80aeed0 0x7f06d814df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d80aeed0 0x7f06d814df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42638/0 (socket says 172.21.15.33:42638) 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 -- 172.21.15.33:0/3686680282 learned_addr learned my addr 172.21.15.33:0/3686680282 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06de030640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06d806b210 0x7f06d814b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06de831640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06d815f680 0x7f06d814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 -- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06d815f680 msgr2=0x7f06d814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06d815f680 0x7f06d814e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 -- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06d806b210 msgr2=0x7f06d814b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06d806b210 0x7f06d814b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 -- 172.21.15.33:0/3686680282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06d806f390 con 0x7f06d80aeed0 2024-04-25T01:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06de831640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06d815f680 0x7f06d814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06dd82f640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d80aeed0 0x7f06d814df40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f06c800b410 tx=0x7f06c800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06ceffd640 1 -- 172.21.15.33:0/3686680282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8015030 con 0x7f06d80aeed0 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06ceffd640 1 -- 172.21.15.33:0/3686680282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06c8004020 con 0x7f06d80aeed0 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06d8155bb0 con 0x7f06d80aeed0 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06ceffd640 1 -- 172.21.15.33:0/3686680282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c8004450 con 0x7f06d80aeed0 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06d806efb0 con 0x7f06d80aeed0 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06d8069040 con 0x7f06d80aeed0 2024-04-25T01:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06ceffd640 1 -- 172.21.15.33:0/3686680282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06c8018070 con 0x7f06d80aeed0 2024-04-25T01:11:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06ceffd640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.375+0000 7f06ceffd640 1 -- 172.21.15.33:0/3686680282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f06c8085b10 con 0x7f06d80aeed0 2024-04-25T01:11:49.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.379+0000 7f06de030640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.379+0000 7f06de030640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f06d40ad000 tx=0x7f06d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.383+0000 7f06ceffd640 1 -- 172.21.15.33:0/3686680282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f06c8056400 con 0x7f06d80aeed0 2024-04-25T01:11:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.571+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f06d8155490 con 0x7f06d80aeed0 2024-04-25T01:11:49.563 INFO:tasks.ceph.mon.a.smithi033.stderr:2024-04-25T01:11:49.571+0000 7fad5470a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:11:49.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.575+0000 7f06ceffd640 1 -- 172.21.15.33:0/3686680282 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f06c8055d00 con 0x7f06d80aeed0 2024-04-25T01:11:49.566 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T01:11:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 msgr2=0x7f06bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f06d40ad000 tx=0x7f06d4008040 comp rx=0 tx=0).stop 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d80aeed0 msgr2=0x7f06d814df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d80aeed0 0x7f06d814df40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f06c800b410 tx=0x7f06c800b8d0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 shutdown_connections 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f06bc0678c0 0x7f06bc069d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f06d815f680 0x7f06d814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f06d80aeed0 0x7f06d814df40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 --2- 172.21.15.33:0/3686680282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06d806b210 0x7f06d814b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 >> 172.21.15.33:0/3686680282 conn(0x7f06d80b50c0 msgr2=0x7f06d80b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 shutdown_connections 2024-04-25T01:11:49.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.583+0000 7f06e0294640 1 -- 172.21.15.33:0/3686680282 wait complete. 2024-04-25T01:11:49.574 INFO:tasks.ceph.mds.c.smithi033.stderr: -7215> 2024-04-25T01:09:49.732+0000 7fc145c08640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T01:11:49.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T01:11:49.620 INFO:tasks.ceph.mds.c.smithi033.stdout:starting mds.c at 2024-04-25T01:11:49.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f677e24a640 1 Processor -- start 2024-04-25T01:11:49.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f677e24a640 1 -- start start 2024-04-25T01:11:49.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f677e24a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6778072410 0x7f67780b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f677e24a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67780b4a20 con 0x7f677815ae20 2024-04-25T01:11:49.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f677e24a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67780b2bb0 con 0x7f677806ecc0 2024-04-25T01:11:49.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f677e24a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f67780b2e90 con 0x7f6778072410 2024-04-25T01:11:49.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f6777fff640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6778072410 0x7f67780b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f67777fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f677815ae20 0x7f677806e5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33110/0 (socket says 172.21.15.33:33110) 2024-04-25T01:11:49.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f67777fe640 1 -- 172.21.15.33:0/1578903398 learned_addr learned my addr 172.21.15.33:0/1578903398 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:49.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f6777fff640 1 -- 172.21.15.33:0/1578903398 >> v1:172.21.15.138:6789/0 conn(0x7f677806ecc0 legacy=0x7f67780cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:49.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f6777fff640 1 -- 172.21.15.33:0/1578903398 >> v1:172.21.15.33:6789/0 conn(0x7f677815ae20 legacy=0x7f677806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.719+0000 7f6777fff640 1 -- 172.21.15.33:0/1578903398 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67780b3170 con 0x7f6778072410 2024-04-25T01:11:49.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f6777fff640 1 --2- 172.21.15.33:0/1578903398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6778072410 0x7f67780b3690 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6770009d90 tx=0x7f677001eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da080fe9268db4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f67767fc640 1 -- 172.21.15.33:0/1578903398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700b4070 con 0x7f6778072410 2024-04-25T01:11:49.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f67767fc640 1 -- 172.21.15.33:0/1578903398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67700a7c10 con 0x7f6778072410 2024-04-25T01:11:49.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f67767fc640 1 -- 172.21.15.33:0/1578903398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67700af440 con 0x7f6778072410 2024-04-25T01:11:49.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 -- 172.21.15.33:0/1578903398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6778072410 msgr2=0x7f67780b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 --2- 172.21.15.33:0/1578903398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6778072410 0x7f67780b3690 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6770009d90 tx=0x7f677001eb40 comp rx=0 tx=0).stop 2024-04-25T01:11:49.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 -- 172.21.15.33:0/1578903398 shutdown_connections 2024-04-25T01:11:49.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 --2- 172.21.15.33:0/1578903398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6778072410 0x7f67780b3690 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 -- 172.21.15.33:0/1578903398 >> 172.21.15.33:0/1578903398 conn(0x7f67780b1810 msgr2=0x7f67780b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:49.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 -- 172.21.15.33:0/1578903398 shutdown_connections 2024-04-25T01:11:49.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 -- 172.21.15.33:0/1578903398 wait complete. 2024-04-25T01:11:49.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 Processor -- start 2024-04-25T01:11:49.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.723+0000 7f677e24a640 1 -- start start 2024-04-25T01:11:49.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677806ecc0 0x7f677815c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6778072410 0x7f6778158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f677815ae20 0x7f6778148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f677806d620 con 0x7f677806ecc0 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f677806d4a0 con 0x7f6778072410 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f677806d7a0 con 0x7f677815ae20 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677806ecc0 0x7f677815c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677806ecc0 0x7f677815c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42676/0 (socket says 172.21.15.33:42676) 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 -- 172.21.15.33:0/439309797 learned_addr learned my addr 172.21.15.33:0/439309797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f6777fff640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f677815ae20 0x7f6778148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f6776ffd640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6778072410 0x7f6778158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 -- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f677815ae20 msgr2=0x7f6778148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f677815ae20 0x7f6778148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 -- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6778072410 msgr2=0x7f6778158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6778072410 0x7f6778158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 -- 172.21.15.33:0/439309797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6778158b50 con 0x7f677806ecc0 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f67777fe640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677806ecc0 0x7f677815c1d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f676800c3a0 tx=0x7f676800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f675ffff640 1 -- 172.21.15.33:0/439309797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768017070 con 0x7f677806ecc0 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f675ffff640 1 -- 172.21.15.33:0/439309797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6768005cf0 con 0x7f677806ecc0 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f675ffff640 1 -- 172.21.15.33:0/439309797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67680072f0 con 0x7f677806ecc0 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6778158df0 con 0x7f677806ecc0 2024-04-25T01:11:49.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67781593b0 con 0x7f677806ecc0 2024-04-25T01:11:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6778069040 con 0x7f677806ecc0 2024-04-25T01:11:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f675ffff640 1 -- 172.21.15.33:0/439309797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f676801a070 con 0x7f677806ecc0 2024-04-25T01:11:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f675ffff640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f675ffff640 1 -- 172.21.15.33:0/439309797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6216+0+0 (secure 0 0 0) 0x7f67680851b0 con 0x7f677806ecc0 2024-04-25T01:11:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.727+0000 7f6776ffd640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:49.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.731+0000 7f6776ffd640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67640678c0 0x7f6764069d80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f677815b440 tx=0x7f676c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:49.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.731+0000 7f675ffff640 1 -- 172.21.15.33:0/439309797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6768055af0 con 0x7f677806ecc0 2024-04-25T01:11:49.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:49.919+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f67781669e0 con 0x7f677806ecc0 2024-04-25T01:11:50.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f675ffff640 1 -- 172.21.15.33:0/439309797 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f67680553f0 con 0x7f677806ecc0 2024-04-25T01:11:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67640678c0 msgr2=0x7f6764069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67640678c0 0x7f6764069d80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f677815b440 tx=0x7f676c009040 comp rx=0 tx=0).stop 2024-04-25T01:11:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677806ecc0 msgr2=0x7f677815c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677806ecc0 0x7f677815c1d0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f676800c3a0 tx=0x7f676800c860 comp rx=0 tx=0).stop 2024-04-25T01:11:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 shutdown_connections 2024-04-25T01:11:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f67640678c0 0x7f6764069d80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f677815ae20 0x7f6778148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6778072410 0x7f6778158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 --2- 172.21.15.33:0/439309797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f677806ecc0 0x7f677815c1d0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 >> 172.21.15.33:0/439309797 conn(0x7f67780b1810 msgr2=0x7f67780aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 shutdown_connections 2024-04-25T01:11:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.579+0000 7f677e24a640 1 -- 172.21.15.33:0/439309797 wait complete. 2024-04-25T01:11:50.588 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T01:11:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.759+0000 7f03cfacb640 1 Processor -- start 2024-04-25T01:11:50.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.759+0000 7f03cfacb640 1 -- start start 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cfacb640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03c81453f0 0x7f03c806e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cfacb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c815ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cfacb640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f03c80b38c0 0x7f03c80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cfacb640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f03c80b3430 con 0x7f03c81453f0 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cfacb640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f03c80b2b40 con 0x7f03c80b38c0 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cfacb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03c80b2de0 con 0x7f03c81457d0 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03c81453f0 0x7f03c806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:50.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f03c81453f0 0x7f03c806e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42690/0 (socket says 172.21.15.33:42690) 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 -- 172.21.15.33:0/15191691 learned_addr learned my addr 172.21.15.33:0/15191691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03ce068640 1 --2- 172.21.15.33:0/15191691 >> v2:172.21.15.138:3300/0 conn(0x7f03c80b38c0 0x7f03c80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd066640 1 --2- 172.21.15.33:0/15191691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c815ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 -- 172.21.15.33:0/15191691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 msgr2=0x7f03c815ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 --2- 172.21.15.33:0/15191691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c815ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 -- 172.21.15.33:0/15191691 >> v2:172.21.15.138:3300/0 conn(0x7f03c80b38c0 msgr2=0x7f03c80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 --2- 172.21.15.33:0/15191691 >> v2:172.21.15.138:3300/0 conn(0x7f03c80b38c0 0x7f03c80cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 -- 172.21.15.33:0/15191691 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c80b30c0 con 0x7f03c81453f0 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03ce068640 1 --2- 172.21.15.33:0/15191691 >> v2:172.21.15.138:3300/0 conn(0x7f03c80b38c0 0x7f03c80cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:50.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd066640 1 --2- 172.21.15.33:0/15191691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cd867640 1 --2- 172.21.15.33:0/15191691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c81453f0 0x7f03c806e870 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f03c8159f00 tx=0x7f03c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f363c0b6667f18d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cc865640 1 -- 172.21.15.33:0/15191691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40b0070 con 0x7f03c81453f0 2024-04-25T01:11:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cc865640 1 -- 172.21.15.33:0/15191691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c40ab600 con 0x7f03c81453f0 2024-04-25T01:11:50.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.763+0000 7f03cc865640 1 -- 172.21.15.33:0/15191691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40ab9d0 con 0x7f03c81453f0 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/15191691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c81453f0 msgr2=0x7f03c806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 --2- 172.21.15.33:0/15191691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c81453f0 0x7f03c806e870 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f03c8159f00 tx=0x7f03c401c520 comp rx=0 tx=0).stop 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/15191691 shutdown_connections 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 --2- 172.21.15.33:0/15191691 >> v2:172.21.15.138:3300/0 conn(0x7f03c80b38c0 0x7f03c80cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 --2- 172.21.15.33:0/15191691 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c815ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 --2- 172.21.15.33:0/15191691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c81453f0 0x7f03c806e870 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/15191691 >> 172.21.15.33:0/15191691 conn(0x7f03c806b3e0 msgr2=0x7f03c806b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/15191691 shutdown_connections 2024-04-25T01:11:50.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/15191691 wait complete. 2024-04-25T01:11:50.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 Processor -- start 2024-04-25T01:11:50.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- start start 2024-04-25T01:11:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c80b38c0 0x7f03c815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03c81453f0 0x7f03c81f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c80af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:50.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81604d0 con 0x7f03c80b38c0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8160350 con 0x7f03c81453f0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f03c8160650 con 0x7f03c81457d0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c80b38c0 0x7f03c815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c80b38c0 0x7f03c815b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42706/0 (socket says 172.21.15.33:42706) 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 -- 172.21.15.33:0/132138416 learned_addr learned my addr 172.21.15.33:0/132138416 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd066640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03c81453f0 0x7f03c81f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03ce068640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c80af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 -- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 msgr2=0x7f03c80af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c80af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 -- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03c81453f0 msgr2=0x7f03c81f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03c81453f0 0x7f03c81f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 -- 172.21.15.33:0/132138416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c815b6d0 con 0x7f03c80b38c0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cd867640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c80b38c0 0x7f03c815b190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f03c40ae040 tx=0x7f03c401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:50.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03be7fc640 1 -- 172.21.15.33:0/132138416 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40b0040 con 0x7f03c80b38c0 2024-04-25T01:11:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03be7fc640 1 -- 172.21.15.33:0/132138416 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c40bc070 con 0x7f03c80b38c0 2024-04-25T01:11:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c815d860 con 0x7f03c80b38c0 2024-04-25T01:11:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03be7fc640 1 -- 172.21.15.33:0/132138416 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c40b6860 con 0x7f03c80b38c0 2024-04-25T01:11:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c815de20 con 0x7f03c80b38c0 2024-04-25T01:11:50.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.767+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c8069040 con 0x7f03c80b38c0 2024-04-25T01:11:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.771+0000 7f03be7fc640 1 -- 172.21.15.33:0/132138416 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f03c40b6a00 con 0x7f03c80b38c0 2024-04-25T01:11:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.771+0000 7f03be7fc640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03ac0678c0 0x7f03ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.771+0000 7f03be7fc640 1 -- 172.21.15.33:0/132138416 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6216+0+0 (secure 0 0 0) 0x7f03c4120ee0 con 0x7f03c80b38c0 2024-04-25T01:11:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.771+0000 7f03cd066640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03ac0678c0 0x7f03ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:50.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.771+0000 7f03cd066640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03ac0678c0 0x7f03ac069d80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006fd0 tx=0x7f03b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.775+0000 7f03be7fc640 1 -- 172.21.15.33:0/132138416 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f03c815b6d0 con 0x7f03c80b38c0 2024-04-25T01:11:50.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:50.935+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f03c80ad9c0 con 0x7f03c80b38c0 2024-04-25T01:11:51.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.583+0000 7f03be7fc640 1 -- 172.21.15.33:0/132138416 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f03c4016110 con 0x7f03c80b38c0 2024-04-25T01:11:51.576 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-04-25T01:11:51.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03ac0678c0 msgr2=0x7f03ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:51.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03ac0678c0 0x7f03ac069d80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f03b8006fd0 tx=0x7f03b8008040 comp rx=0 tx=0).stop 2024-04-25T01:11:51.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c80b38c0 msgr2=0x7f03c815b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c80b38c0 0x7f03c815b190 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f03c40ae040 tx=0x7f03c401ad60 comp rx=0 tx=0).stop 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 shutdown_connections 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f03ac0678c0 0x7f03ac069d80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f03c81457d0 0x7f03c80af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03c81453f0 0x7f03c81f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 --2- 172.21.15.33:0/132138416 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03c80b38c0 0x7f03c815b190 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 >> 172.21.15.33:0/132138416 conn(0x7f03c806b3e0 msgr2=0x7f03c80b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:51.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 shutdown_connections 2024-04-25T01:11:51.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.591+0000 7f03cfacb640 1 -- 172.21.15.33:0/132138416 wait complete. 2024-04-25T01:11:51.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T01:11:51.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba8c05a640 1 Processor -- start 2024-04-25T01:11:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba8c05a640 1 -- start start 2024-04-25T01:11:51.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba8c05a640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba84144700 0x7fba8415e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba8c05a640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8415efa0 con 0x7fba8406ca40 2024-04-25T01:11:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba8c05a640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba8415f280 con 0x7fba840cfc10 2024-04-25T01:11:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba8c05a640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fba84072040 con 0x7fba84144700 2024-04-25T01:11:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba89df6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fba84144700 0x7fba8415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba895f5640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fba8406ca40 0x7fba840727b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33116/0 (socket says 172.21.15.33:33116) 2024-04-25T01:11:51.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.739+0000 7fba895f5640 1 -- 172.21.15.33:0/1924993573 learned_addr learned my addr 172.21.15.33:0/1924993573 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba89df6640 1 -- 172.21.15.33:0/1924993573 >> v1:172.21.15.138:6789/0 conn(0x7fba840cfc10 legacy=0x7fba840cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba89df6640 1 -- 172.21.15.33:0/1924993573 >> v1:172.21.15.33:6789/0 conn(0x7fba8406ca40 legacy=0x7fba840727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba89df6640 1 -- 172.21.15.33:0/1924993573 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba8415f560 con 0x7fba84144700 2024-04-25T01:11:51.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba88df4640 1 -- 172.21.15.33:0/1924993573 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902443298 0 0) 0x7fba8415efa0 con 0x7fba8406ca40 2024-04-25T01:11:51.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba89df6640 1 --2- 172.21.15.33:0/1924993573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba8415e770 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba800097c0 tx=0x7fba8001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34ead8e17909a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:51.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba88df4640 1 -- 172.21.15.33:0/1924993573 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba800b1070 con 0x7fba84144700 2024-04-25T01:11:51.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba88df4640 1 -- 172.21.15.33:0/1924993573 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba800ac640 con 0x7fba84144700 2024-04-25T01:11:51.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba88df4640 1 -- 172.21.15.33:0/1924993573 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba800aca10 con 0x7fba84144700 2024-04-25T01:11:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba8c05a640 1 -- 172.21.15.33:0/1924993573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 msgr2=0x7fba8415e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba8c05a640 1 --2- 172.21.15.33:0/1924993573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba8415e770 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fba800097c0 tx=0x7fba8001c430 comp rx=0 tx=0).stop 2024-04-25T01:11:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba8c05a640 1 -- 172.21.15.33:0/1924993573 shutdown_connections 2024-04-25T01:11:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba8c05a640 1 --2- 172.21.15.33:0/1924993573 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba8415e770 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:51.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba8c05a640 1 -- 172.21.15.33:0/1924993573 >> 172.21.15.33:0/1924993573 conn(0x7fba8406b3e0 msgr2=0x7fba8406b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba8c05a640 1 -- 172.21.15.33:0/1924993573 shutdown_connections 2024-04-25T01:11:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.743+0000 7fba8c05a640 1 -- 172.21.15.33:0/1924993573 wait complete. 2024-04-25T01:11:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 Processor -- start 2024-04-25T01:11:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 -- start start 2024-04-25T01:11:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba8406ca40 0x7fba84158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:51.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba840cfc10 0x7fba841388c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba840b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba840b33b0 con 0x7fba84144700 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fba840d0450 con 0x7fba840cfc10 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fba840b3530 con 0x7fba8406ca40 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba840b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba840b09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42722/0 (socket says 172.21.15.33:42722) 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 -- 172.21.15.33:0/3470233473 learned_addr learned my addr 172.21.15.33:0/3470233473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba895f5640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba840cfc10 0x7fba841388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba89df6640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba8406ca40 0x7fba84158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 -- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba8406ca40 msgr2=0x7fba84158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba8406ca40 0x7fba84158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 -- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba840cfc10 msgr2=0x7fba841388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:51.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba840cfc10 0x7fba841388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 -- 172.21.15.33:0/3470233473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba84158bc0 con 0x7fba84144700 2024-04-25T01:11:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8a5f7640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba840b09c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c780 tx=0x7fba7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba7a7fc640 1 -- 172.21.15.33:0/3470233473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c004b70 con 0x7fba84144700 2024-04-25T01:11:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba7a7fc640 1 -- 172.21.15.33:0/3470233473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba7c004d10 con 0x7fba84144700 2024-04-25T01:11:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba84169a70 con 0x7fba84144700 2024-04-25T01:11:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba7a7fc640 1 -- 172.21.15.33:0/3470233473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba7c00aa90 con 0x7fba84144700 2024-04-25T01:11:51.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba8416a030 con 0x7fba84144700 2024-04-25T01:11:51.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba84069040 con 0x7fba84144700 2024-04-25T01:11:51.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.747+0000 7fba7a7fc640 1 -- 172.21.15.33:0/3470233473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba7c00ac30 con 0x7fba84144700 2024-04-25T01:11:51.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.751+0000 7fba7a7fc640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba680678c0 0x7fba68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:51.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.751+0000 7fba7a7fc640 1 -- 172.21.15.33:0/3470233473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5778+0+0 (secure 0 0 0) 0x7fba7c085f60 con 0x7fba84144700 2024-04-25T01:11:51.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.751+0000 7fba89df6640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba680678c0 0x7fba68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:51.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.751+0000 7fba89df6640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba680678c0 0x7fba68069d80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fba800ae000 tx=0x7fba800a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:51.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.755+0000 7fba7a7fc640 1 -- 172.21.15.33:0/3470233473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fba7c020070 con 0x7fba84144700 2024-04-25T01:11:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:51.915+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fba84071940 con 0x7fba84144700 2024-04-25T01:11:52.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.587+0000 7fba7a7fc640 1 -- 172.21.15.33:0/3470233473 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7fba7c056a50 con 0x7fba84144700 2024-04-25T01:11:52.579 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-04-25T01:11:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba680678c0 msgr2=0x7fba68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba680678c0 0x7fba68069d80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fba800ae000 tx=0x7fba800a6070 comp rx=0 tx=0).stop 2024-04-25T01:11:52.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 msgr2=0x7fba840b09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba840b09c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fba7c00c780 tx=0x7fba7c00cc40 comp rx=0 tx=0).stop 2024-04-25T01:11:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 shutdown_connections 2024-04-25T01:11:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fba680678c0 0x7fba68069d80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba84144700 0x7fba840b09c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fba840cfc10 0x7fba841388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 --2- 172.21.15.33:0/3470233473 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fba8406ca40 0x7fba84158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 >> 172.21.15.33:0/3470233473 conn(0x7fba8406b3e0 msgr2=0x7fba8406de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:52.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 shutdown_connections 2024-04-25T01:11:52.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.595+0000 7fba8c05a640 1 -- 172.21.15.33:0/3470233473 wait complete. 2024-04-25T01:11:52.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T01:11:52.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.739+0000 7f753e374640 1 Processor -- start 2024-04-25T01:11:52.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.739+0000 7f753e374640 1 -- start start 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f753e374640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 0x7f75380cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f753e374640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75380b6cc0 con 0x7f75380d2ac0 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f753e374640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75380b6f30 con 0x7f75380d1f60 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f753e374640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f75380b7580 con 0x7f753815e0d0 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f753c911640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f75380d2ac0 0x7f7538167cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33132/0 (socket says 172.21.15.33:33132) 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f753c911640 1 -- 172.21.15.33:0/127688504 learned_addr learned my addr 172.21.15.33:0/127688504 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7537fff640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 0x7f75380cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4039276691 0 0) 0x7f75380b6cc0 con 0x7f75380d2ac0 2024-04-25T01:11:52.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7524003530 con 0x7f75380d2ac0 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3016480607 0 0) 0x7f7524003530 con 0x7f75380d2ac0 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 msgr2=0x7f75380cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 0x7f75380cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 >> v1:172.21.15.138:6789/0 conn(0x7f75380d1f60 legacy=0x7f75380d23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75380b7210 con 0x7f75380d2ac0 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f7530001a00 con 0x7f75380d2ac0 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 >> v1:172.21.15.33:6789/0 conn(0x7f75380d2ac0 legacy=0x7f7538167cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 0x7f75240039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7524003f70 0x7f7524004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7524004d50 0x7f75240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75380b7580 con 0x7f7524003f70 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f752400bab0 con 0x7f7524004d50 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f752400bcb0 con 0x7f753815e0d0 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f75377fe640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7524003f70 0x7f7524004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f753c911640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7524004d50 0x7f75240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f7537fff640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 0x7f75240039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f75377fe640 1 -- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 msgr2=0x7f75240039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f75377fe640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f753815e0d0 0x7f75240039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f75377fe640 1 -- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7524004d50 msgr2=0x7f75240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f75377fe640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7524004d50 0x7f75240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.743+0000 7f75377fe640 1 -- 172.21.15.33:0/127688504 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7524003530 con 0x7f7524003f70 2024-04-25T01:11:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f75377fe640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7524003f70 0x7f7524004810 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7528001750 tx=0x7f752801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb48055c9da90f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:52.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75280013a0 con 0x7f7524003f70 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75280052f0 con 0x7f7524003f70 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f7536ffd640 1 -- 172.21.15.33:0/127688504 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528003840 con 0x7f7524003f70 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- 172.21.15.33:0/127688504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7524003f70 msgr2=0x7f7524004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 --2- 172.21.15.33:0/127688504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7524003f70 0x7f7524004810 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7528001750 tx=0x7f752801c3e0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 -- 172.21.15.33:0/127688504 reap_dead start 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- 172.21.15.33:0/127688504 shutdown_connections 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- 172.21.15.33:0/127688504 >> 172.21.15.33:0/127688504 conn(0x7f75380d3dd0 msgr2=0x7f75380d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- 172.21.15.33:0/127688504 shutdown_connections 2024-04-25T01:11:52.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- 172.21.15.33:0/127688504 wait complete. 2024-04-25T01:11:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 Processor -- start 2024-04-25T01:11:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- start start 2024-04-25T01:11:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7524003f70 0x7f75381f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f753815fe80 0x7f75380c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538160300 0x7f75380c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f753815e630 con 0x7f7538160300 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f753815e4b0 con 0x7f753815fe80 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f75380cf3f0 con 0x7f7524003f70 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538160300 0x7f75380c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538160300 0x7f75380c7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42738/0 (socket says 172.21.15.33:42738) 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 -- 172.21.15.33:0/966188264 learned_addr learned my addr 172.21.15.33:0/966188264 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f7537fff640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7524003f70 0x7f75381f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f75377fe640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f753815fe80 0x7f75380c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 -- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7524003f70 msgr2=0x7f75381f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7524003f70 0x7f75381f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 -- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f753815fe80 msgr2=0x7f75380c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f753815fe80 0x7f75380c7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 -- 172.21.15.33:0/966188264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75380c83f0 con 0x7f7538160300 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f75377fe640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f753815fe80 0x7f75380c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753c911640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538160300 0x7f75380c7c70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f75380b5890 tx=0x7f7530011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:52.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f7534ff9640 1 -- 172.21.15.33:0/966188264 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530010040 con 0x7f7538160300 2024-04-25T01:11:52.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f7534ff9640 1 -- 172.21.15.33:0/966188264 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f753001a070 con 0x7f7538160300 2024-04-25T01:11:52.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f7534ff9640 1 -- 172.21.15.33:0/966188264 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753000fba0 con 0x7f7538160300 2024-04-25T01:11:52.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75380c9080 con 0x7f7538160300 2024-04-25T01:11:52.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.747+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75381f4580 con 0x7f7538160300 2024-04-25T01:11:52.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.751+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7538069040 con 0x7f7538160300 2024-04-25T01:11:52.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.751+0000 7f7534ff9640 1 -- 172.21.15.33:0/966188264 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f753000fd40 con 0x7f7538160300 2024-04-25T01:11:52.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.751+0000 7f7534ff9640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f75180678c0 0x7f7518069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:52.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.751+0000 7f7534ff9640 1 -- 172.21.15.33:0/966188264 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f7530088b00 con 0x7f7538160300 2024-04-25T01:11:52.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.751+0000 7f7537fff640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f75180678c0 0x7f7518069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:52.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.751+0000 7f7537fff640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f75180678c0 0x7f7518069d80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7520009a30 tx=0x7f7520008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:52.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.755+0000 7f7534ff9640 1 -- 172.21.15.33:0/966188264 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f7530059710 con 0x7f7538160300 2024-04-25T01:11:52.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.927+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f75380c7510 con 0x7f7538160300 2024-04-25T01:11:52.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.931+0000 7f7534ff9640 1 -- 172.21.15.33:0/966188264 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f7530088e20 con 0x7f7538160300 2024-04-25T01:11:52.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f75180678c0 msgr2=0x7f7518069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f75180678c0 0x7f7518069d80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7520009a30 tx=0x7f7520008040 comp rx=0 tx=0).stop 2024-04-25T01:11:52.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538160300 msgr2=0x7f75380c7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:52.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538160300 0x7f75380c7c70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f75380b5890 tx=0x7f7530011c20 comp rx=0 tx=0).stop 2024-04-25T01:11:52.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 shutdown_connections 2024-04-25T01:11:52.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f75180678c0 0x7f7518069d80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7538160300 0x7f75380c7c70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f753815fe80 0x7f75380c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 --2- 172.21.15.33:0/966188264 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f7524003f70 0x7f75381f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:52.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 >> 172.21.15.33:0/966188264 conn(0x7f75380d3dd0 msgr2=0x7f753806bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:52.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 shutdown_connections 2024-04-25T01:11:52.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:52.935+0000 7f753e374640 1 -- 172.21.15.33:0/966188264 wait complete. 2024-04-25T01:11:52.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_acls.TestACLs.test_acls' 2024-04-25T01:11:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb8511ac640 1 Processor -- start 2024-04-25T01:11:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb8511ac640 1 -- start start 2024-04-25T01:11:53.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb8511ac640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb84c15c160 0x7fb84c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb8511ac640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84c15f9a0 con 0x7fb84c0b5360 2024-04-25T01:11:53.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb8511ac640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb84c15fc80 con 0x7fb84c145470 2024-04-25T01:11:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb8511ac640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fb84c0b57b0 con 0x7fb84c15c160 2024-04-25T01:11:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84b59e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fb84c0b5360 0x7fb84c0cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33136/0 (socket says 172.21.15.33:33136) 2024-04-25T01:11:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84b59e640 1 -- 172.21.15.33:0/3534999818 learned_addr learned my addr 172.21.15.33:0/3534999818 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84a59c640 1 --2- 172.21.15.33:0/3534999818 >> v2:172.21.15.138:3300/0 conn(0x7fb84c15c160 0x7fb84c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2837509519 0 0) 0x7fb84c15f9a0 con 0x7fb84c0b5360 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb838003530 con 0x7fb84c0b5360 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 539150886 0 0) 0x7fb838003530 con 0x7fb84c0b5360 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 >> v2:172.21.15.138:3300/0 conn(0x7fb84c15c160 msgr2=0x7fb84c0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 --2- 172.21.15.33:0/3534999818 >> v2:172.21.15.138:3300/0 conn(0x7fb84c15c160 0x7fb84c0b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 >> v1:172.21.15.138:6789/0 conn(0x7fb84c145470 legacy=0x7fb84c0cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c15ff60 con 0x7fb84c0b5360 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fb83c001680 con 0x7fb84c0b5360 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 >> v1:172.21.15.33:6789/0 conn(0x7fb84c0b5360 legacy=0x7fb84c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8380039c0 0x7fb838003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838004810 0x7fb838004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380051d0 0x7fb83800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c0b57b0 con 0x7fb8380039c0 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb83800c070 con 0x7fb838004810 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb83800c270 con 0x7fb8380051d0 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8380039c0 0x7fb838003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84a59c640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838004810 0x7fb838004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84b59e640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380051d0 0x7fb83800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84ad9d640 1 -- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380051d0 msgr2=0x7fb83800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380051d0 0x7fb83800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84ad9d640 1 -- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838004810 msgr2=0x7fb838004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838004810 0x7fb838004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84b59e640 1 -- 172.21.15.33:0/3534999818 reap_dead start 2024-04-25T01:11:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84ad9d640 1 -- 172.21.15.33:0/3534999818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb838003530 con 0x7fb8380039c0 2024-04-25T01:11:53.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.075+0000 7fb84a59c640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb838004810 0x7fb838004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8380039c0 0x7fb838003de0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb844001780 tx=0x7fb84401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b31fe2d5186da54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440a8050 con 0x7fb8380039c0 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8440052f0 con 0x7fb8380039c0 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb849d9b640 1 -- 172.21.15.33:0/3534999818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8440013a0 con 0x7fb8380039c0 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb8511ac640 1 -- 172.21.15.33:0/3534999818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8380039c0 msgr2=0x7fb838003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb8511ac640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8380039c0 0x7fb838003de0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb844001780 tx=0x7fb84401c400 comp rx=0 tx=0).stop 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb8511ac640 1 -- 172.21.15.33:0/3534999818 shutdown_connections 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb8511ac640 1 --2- 172.21.15.33:0/3534999818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8380039c0 0x7fb838003de0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb8511ac640 1 -- 172.21.15.33:0/3534999818 >> 172.21.15.33:0/3534999818 conn(0x7fb84c0b1810 msgr2=0x7fb84c0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb8511ac640 1 -- 172.21.15.33:0/3534999818 shutdown_connections 2024-04-25T01:11:53.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.079+0000 7fb8511ac640 1 -- 172.21.15.33:0/3534999818 wait complete. 2024-04-25T01:11:53.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 Processor -- start 2024-04-25T01:11:53.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 -- start start 2024-04-25T01:11:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380039c0 0x7fb84c13a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb84c13eee0 0x7fb84c15e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb84c15ecd0 0x7fb84c0adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c06e850 con 0x7fb84c15ecd0 2024-04-25T01:11:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb84c15f750 con 0x7fb84c13eee0 2024-04-25T01:11:53.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb84c071b90 con 0x7fb8380039c0 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb84c15ecd0 0x7fb84c0adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb84c15ecd0 0x7fb84c0adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42762/0 (socket says 172.21.15.33:42762) 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 -- 172.21.15.33:0/613411839 learned_addr learned my addr 172.21.15.33:0/613411839 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84a59c640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb84c13eee0 0x7fb84c15e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380039c0 0x7fb84c13a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 -- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380039c0 msgr2=0x7fb84c13a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380039c0 0x7fb84c13a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 -- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb84c13eee0 msgr2=0x7fb84c15e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb84c13eee0 0x7fb84c15e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 -- 172.21.15.33:0/613411839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84c0ae880 con 0x7fb84c15ecd0 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380039c0 0x7fb84c13a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb84b59e640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb84c15ecd0 0x7fb84c0adb50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c10 tx=0x7fb83c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8437fe640 1 -- 172.21.15.33:0/613411839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0075a0 con 0x7fb84c15ecd0 2024-04-25T01:11:53.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8437fe640 1 -- 172.21.15.33:0/613411839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb83c0047f0 con 0x7fb84c15ecd0 2024-04-25T01:11:53.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84c13c370 con 0x7fb84c15ecd0 2024-04-25T01:11:53.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8437fe640 1 -- 172.21.15.33:0/613411839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83c004be0 con 0x7fb84c15ecd0 2024-04-25T01:11:53.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84c13c930 con 0x7fb84c15ecd0 2024-04-25T01:11:53.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb84c069040 con 0x7fb84c15ecd0 2024-04-25T01:11:53.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.083+0000 7fb8437fe640 1 -- 172.21.15.33:0/613411839 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb83c014430 con 0x7fb84c15ecd0 2024-04-25T01:11:53.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.087+0000 7fb8437fe640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb82c0678f0 0x7fb82c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:53.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.087+0000 7fb8437fe640 1 -- 172.21.15.33:0/613411839 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fb83c0855a0 con 0x7fb84c15ecd0 2024-04-25T01:11:53.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.087+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb82c0678f0 0x7fb82c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:53.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.087+0000 7fb84ad9d640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb82c0678f0 0x7fb82c069db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb844001750 tx=0x7fb8440011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:53.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.091+0000 7fb8437fe640 1 -- 172.21.15.33:0/613411839 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb83c0561b0 con 0x7fb84c15ecd0 2024-04-25T01:11:53.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.311+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]} v 0) -- 0x7fb84c071420 con 0x7fb84c15ecd0 2024-04-25T01:11:53.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8437fe640 1 -- 172.21.15.33:0/613411839 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_acls.TestACLs.test_acls"]}]=0 v165) ==== 120+0+0 (secure 0 0 0) 0x7fb83c055ab0 con 0x7fb84c15ecd0 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb82c0678f0 msgr2=0x7fb82c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb82c0678f0 0x7fb82c069db0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb844001750 tx=0x7fb8440011a0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb84c15ecd0 msgr2=0x7fb84c0adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb84c15ecd0 0x7fb84c0adb50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c10 tx=0x7fb83c00d9d0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 shutdown_connections 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb82c0678f0 0x7fb82c069db0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb84c15ecd0 0x7fb84c0adb50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb84c13eee0 0x7fb84c15e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 --2- 172.21.15.33:0/613411839 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb8380039c0 0x7fb84c13a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.591+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 >> 172.21.15.33:0/613411839 conn(0x7fb84c0b1810 msgr2=0x7fb84c15c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.595+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 shutdown_connections 2024-04-25T01:11:53.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:53.595+0000 7fb8511ac640 1 -- 172.21.15.33:0/613411839 wait complete. 2024-04-25T01:11:53.604 INFO:tasks.cephfs_test_runner:test_acls (tasks.cephfs.test_acls.TestACLs) ... ok 2024-04-25T01:11:53.605 INFO:tasks.cephfs_test_runner: 2024-04-25T01:11:53.605 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T01:11:53.605 INFO:tasks.cephfs_test_runner:Ran 1 test in 162.014s 2024-04-25T01:11:53.605 INFO:tasks.cephfs_test_runner: 2024-04-25T01:11:53.605 INFO:tasks.cephfs_test_runner:OK 2024-04-25T01:11:53.605 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T01:11:53.617 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T01:11:53.631 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T01:11:53.631 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:11:53.631 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:53.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18498 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T01:11:53.637 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:11:53.637 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:53.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18498 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T01:11:53.685 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:11:53.685 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:53.692 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16945 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T01:11:53.692 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:11:53.692 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T01:11:53.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16945 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T01:11:53.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:53.740 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:11:53.746 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T01:11:53.746 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T01:11:53.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:53.747 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T01:11:53.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:53.802 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:11:53.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:53.885 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-25T01:11:54.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:54.079 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T01:11:54.086 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T01:11:54.086 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T01:11:54.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:54.086 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T01:11:54.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:54.139 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:11:54.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:11:54.212 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T01:11:54.358 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T01:11:54.367 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T01:11:54.367 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:11:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90e1acd640 1 Processor -- start 2024-04-25T01:11:54.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90e1acd640 1 -- start start 2024-04-25T01:11:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90e1acd640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f90dc147640 0x7f90dc06dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90e1acd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90dc147a20 con 0x7f90dc15ebf0 2024-04-25T01:11:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90e1acd640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90dc161a20 con 0x7f90dc06dff0 2024-04-25T01:11:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90e1acd640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f90dc1627e0 con 0x7f90dc147640 2024-04-25T01:11:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90dbfff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f90dc15ebf0 0x7f90dc0b4840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33152/0 (socket says 172.21.15.33:33152) 2024-04-25T01:11:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90dbfff640 1 -- 172.21.15.33:0/2094255126 learned_addr learned my addr 172.21.15.33:0/2094255126 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:54.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90db7fe640 1 --2- 172.21.15.33:0/2094255126 >> v2:172.21.15.138:3300/0 conn(0x7f90dc147640 0x7f90dc06dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90da7fc640 1 -- 172.21.15.33:0/2094255126 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2715173313 0 0) 0x7f90dc147a20 con 0x7f90dc15ebf0 2024-04-25T01:11:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90da7fc640 1 -- 172.21.15.33:0/2094255126 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90c4003530 con 0x7f90dc15ebf0 2024-04-25T01:11:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90db7fe640 1 -- 172.21.15.33:0/2094255126 >> v1:172.21.15.138:6789/0 conn(0x7f90dc06dff0 legacy=0x7f90dc15e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90db7fe640 1 -- 172.21.15.33:0/2094255126 >> v1:172.21.15.33:6789/0 conn(0x7f90dc15ebf0 legacy=0x7f90dc0b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.511+0000 7f90db7fe640 1 -- 172.21.15.33:0/2094255126 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc161d00 con 0x7f90dc147640 2024-04-25T01:11:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90db7fe640 1 --2- 172.21.15.33:0/2094255126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc06dab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009d90 tx=0x7f90d401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddbef7ccd21ea11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90da7fc640 1 -- 172.21.15.33:0/2094255126 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d40b2070 con 0x7f90dc147640 2024-04-25T01:11:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90da7fc640 1 -- 172.21.15.33:0/2094255126 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d40a57a0 con 0x7f90dc147640 2024-04-25T01:11:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90da7fc640 1 -- 172.21.15.33:0/2094255126 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d40a5b70 con 0x7f90dc147640 2024-04-25T01:11:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 -- 172.21.15.33:0/2094255126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 msgr2=0x7f90dc06dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 --2- 172.21.15.33:0/2094255126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc06dab0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f90d4009d90 tx=0x7f90d401c8e0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 -- 172.21.15.33:0/2094255126 shutdown_connections 2024-04-25T01:11:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 --2- 172.21.15.33:0/2094255126 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc06dab0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 -- 172.21.15.33:0/2094255126 >> 172.21.15.33:0/2094255126 conn(0x7f90dc0b1810 msgr2=0x7f90dc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 -- 172.21.15.33:0/2094255126 shutdown_connections 2024-04-25T01:11:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 -- 172.21.15.33:0/2094255126 wait complete. 2024-04-25T01:11:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 Processor -- start 2024-04-25T01:11:54.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.515+0000 7f90e1acd640 1 -- start start 2024-04-25T01:11:54.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90dc06dff0 0x7f90dc13c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc13e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90dc15ebf0 0x7f90dc15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0b58e0 con 0x7f90dc15ebf0 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc0b5760 con 0x7f90dc147640 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f90dc0b5a60 con 0x7f90dc06dff0 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90dc15ebf0 0x7f90dc15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90dc15ebf0 0x7f90dc15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42774/0 (socket says 172.21.15.33:42774) 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 -- 172.21.15.33:0/1945083489 learned_addr learned my addr 172.21.15.33:0/1945083489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90daffd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc13e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90db7fe640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90dc06dff0 0x7f90dc13c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 -- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90dc06dff0 msgr2=0x7f90dc13c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90dc06dff0 0x7f90dc13c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 -- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 msgr2=0x7f90dc13e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc13e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 -- 172.21.15.33:0/1945083489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc136a10 con 0x7f90dc15ebf0 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90db7fe640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90dc06dff0 0x7f90dc13c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90daffd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc13e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90dbfff640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90dc15ebf0 0x7f90dc15a950 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00c480 tx=0x7f90cc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90c3fff640 1 -- 172.21.15.33:0/1945083489 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc017070 con 0x7f90dc15ebf0 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90c3fff640 1 -- 172.21.15.33:0/1945083489 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90cc005cf0 con 0x7f90dc15ebf0 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc136cf0 con 0x7f90dc15ebf0 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90c3fff640 1 -- 172.21.15.33:0/1945083489 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc0072b0 con 0x7f90dc15ebf0 2024-04-25T01:11:54.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc1372b0 con 0x7f90dc15ebf0 2024-04-25T01:11:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc069040 con 0x7f90dc15ebf0 2024-04-25T01:11:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90c3fff640 1 -- 172.21.15.33:0/1945083489 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f90cc01a070 con 0x7f90dc15ebf0 2024-04-25T01:11:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90c3fff640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90c80678c0 0x7f90c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90c3fff640 1 -- 172.21.15.33:0/1945083489 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f90cc084dd0 con 0x7f90dc15ebf0 2024-04-25T01:11:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.519+0000 7f90db7fe640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90c80678c0 0x7f90c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.523+0000 7f90db7fe640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90c80678c0 0x7f90c8069d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90d40af000 tx=0x7f90d40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.523+0000 7f90c3fff640 1 -- 172.21.15.33:0/1945083489 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f90cc055b10 con 0x7f90dc15ebf0 2024-04-25T01:11:54.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.679+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90dc072780 con 0x7f90c80678c0 2024-04-25T01:11:54.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.679+0000 7f90c3fff640 1 -- 172.21.15.33:0/1945083489 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7f90dc072780 con 0x7f90c80678c0 2024-04-25T01:11:54.671 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:54.671 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:11:54.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90c80678c0 msgr2=0x7f90c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90c80678c0 0x7f90c8069d80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f90d40af000 tx=0x7f90d40a7070 comp rx=0 tx=0).stop 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90dc15ebf0 msgr2=0x7f90dc15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90dc15ebf0 0x7f90dc15a950 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f90cc00c480 tx=0x7f90cc00c940 comp rx=0 tx=0).stop 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 shutdown_connections 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f90c80678c0 0x7f90c8069d80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90dc15ebf0 0x7f90dc15a950 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90dc147640 0x7f90dc13e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 --2- 172.21.15.33:0/1945083489 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f90dc06dff0 0x7f90dc13c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 >> 172.21.15.33:0/1945083489 conn(0x7f90dc0b1810 msgr2=0x7f90dc0ad840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 shutdown_connections 2024-04-25T01:11:54.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.683+0000 7f90e1acd640 1 -- 172.21.15.33:0/1945083489 wait complete. 2024-04-25T01:11:54.688 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-04-25T01:11:53.163057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":229516,"kb_used_data":5024,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745204,"statfs":{"total":773094113280,"available":772859088896,"internally_reserved":0,"allocated":5144576,"data_stored":2911316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7677,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002362"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T01:11:52.781200+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:11:52.781200+0000","last_peered":"2024-04-25T01:11:52.781200+0000","last_clean":"2024-04-25T01:11:52.781200+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:11:52.781200+0000","last_undegraded":"2024-04-25T01:11:52.781200+0000","last_fullsized":"2024-04-25T01:11:52.781200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":814200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":212016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":217858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31668,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":499712,"data_stored":218504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":211524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":211546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1077248,"data_stored":801836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:11:54.689 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:11:54.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc553266640 1 Processor -- start 2024-04-25T01:11:54.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc553266640 1 -- start start 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc553266640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 0x7fc54c0d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc553266640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc54c15e470 0x7fc54c0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc553266640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc54c0d10b0 con 0x7fc54c0d48d0 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc553266640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fc54c0d1de0 con 0x7fc54c15e470 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc553266640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0d1390 con 0x7fc54c0d03e0 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fc54c0d48d0 0x7fc54c0d27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33162/0 (socket says 172.21.15.33:33162) 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 -- 172.21.15.33:0/4245745938 learned_addr learned my addr 172.21.15.33:0/4245745938 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc550801640 1 --2- 172.21.15.33:0/4245745938 >> v2:172.21.15.138:3300/0 conn(0x7fc54c15e470 0x7fc54c0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323455043 0 0) 0x7fc54c0d10b0 con 0x7fc54c0d48d0 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551002640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 0x7fc54c0d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc538003530 con 0x7fc54c0d48d0 2024-04-25T01:11:54.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 65212626 0 0) 0x7fc538003530 con 0x7fc54c0d48d0 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 msgr2=0x7fc54c0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 0x7fc54c0d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 >> v2:172.21.15.138:3300/0 conn(0x7fc54c15e470 msgr2=0x7fc54c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 --2- 172.21.15.33:0/4245745938 >> v2:172.21.15.138:3300/0 conn(0x7fc54c15e470 0x7fc54c0d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0d1670 con 0x7fc54c0d48d0 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fc544001a00 con 0x7fc54c0d48d0 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 >> v1:172.21.15.33:6789/0 conn(0x7fc54c0d48d0 legacy=0x7fc54c0d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 0x7fc5380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538003f40 0x7fc538004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc538004d50 0x7fc5380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0d1de0 con 0x7fc538004d50 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0d1390 con 0x7fc538003f40 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc53800ba20 con 0x7fc54c0d03e0 2024-04-25T01:11:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc538004d50 0x7fc5380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc550801640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538003f40 0x7fc538004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551002640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 0x7fc5380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 -- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 msgr2=0x7fc5380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0d03e0 0x7fc5380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 -- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538003f40 msgr2=0x7fc538004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc538003f40 0x7fc538004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.811+0000 7fc551803640 1 -- 172.21.15.33:0/4245745938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc538003530 con 0x7fc538004d50 2024-04-25T01:11:54.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc551803640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc538004d50 0x7fc5380051d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc54400a890 tx=0x7fc54401fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0966ea2061ca9bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:54.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54400a4e0 con 0x7fc538004d50 2024-04-25T01:11:54.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5440ad020 con 0x7fc538004d50 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc543fff640 1 -- 172.21.15.33:0/4245745938 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5440a9720 con 0x7fc538004d50 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 -- 172.21.15.33:0/4245745938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc538004d50 msgr2=0x7fc5380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 --2- 172.21.15.33:0/4245745938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc538004d50 0x7fc5380051d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc54400a890 tx=0x7fc54401fb00 comp rx=0 tx=0).stop 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc551803640 1 -- 172.21.15.33:0/4245745938 reap_dead start 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 -- 172.21.15.33:0/4245745938 shutdown_connections 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 -- 172.21.15.33:0/4245745938 >> 172.21.15.33:0/4245745938 conn(0x7fc54c0d3db0 msgr2=0x7fc54c0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 -- 172.21.15.33:0/4245745938 shutdown_connections 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 -- 172.21.15.33:0/4245745938 wait complete. 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 Processor -- start 2024-04-25T01:11:54.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 -- start start 2024-04-25T01:11:54.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0a0b60 0x7fc54c09e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.815+0000 7fc553266640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc54c0c9e40 0x7fc54c0b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc553266640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c09ee50 0x7fc54c0b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc553266640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0cfb50 con 0x7fc54c0c9e40 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc553266640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c0cf9d0 con 0x7fc54c09ee50 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc553266640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fc54c0cfcd0 con 0x7fc54c0a0b60 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc54c0c9e40 0x7fc54c0b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc54c0c9e40 0x7fc54c0b68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42796/0 (socket says 172.21.15.33:42796) 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 -- 172.21.15.33:0/2949525201 learned_addr learned my addr 172.21.15.33:0/2949525201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc551002640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0a0b60 0x7fc54c09e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc551803640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c09ee50 0x7fc54c0b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 -- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0a0b60 msgr2=0x7fc54c09e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0a0b60 0x7fc54c09e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 -- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c09ee50 msgr2=0x7fc54c0b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c09ee50 0x7fc54c0b6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 -- 172.21.15.33:0/2949525201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c0c7e50 con 0x7fc54c0c9e40 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc550801640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc54c0c9e40 0x7fc54c0b68f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc53c011690 tx=0x7fc53c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc541ffb640 1 -- 172.21.15.33:0/2949525201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c00f9c0 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc541ffb640 1 -- 172.21.15.33:0/2949525201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc53c00fb60 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc541ffb640 1 -- 172.21.15.33:0/2949525201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc53c015460 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c0b7550 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c0b7b10 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c069040 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc541ffb640 1 -- 172.21.15.33:0/2949525201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc53c015600 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc541ffb640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc5300678f0 0x7fc530069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc541ffb640 1 -- 172.21.15.33:0/2949525201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fc53c086440 con 0x7fc54c0c9e40 2024-04-25T01:11:54.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc551002640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc5300678f0 0x7fc530069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:54.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.819+0000 7fc551002640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc5300678f0 0x7fc530069db0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc548009a30 tx=0x7fc548008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:54.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.823+0000 7fc541ffb640 1 -- 172.21.15.33:0/2949525201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fc53c019030 con 0x7fc54c0c9e40 2024-04-25T01:11:54.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.983+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc54c0ca340 con 0x7fc5300678f0 2024-04-25T01:11:54.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc541ffb640 1 -- 172.21.15.33:0/2949525201 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7fc548013a20 con 0x7fc5300678f0 2024-04-25T01:11:54.977 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:54.977 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:11:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc5300678f0 msgr2=0x7fc530069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc5300678f0 0x7fc530069db0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc548009a30 tx=0x7fc548008040 comp rx=0 tx=0).stop 2024-04-25T01:11:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc54c0c9e40 msgr2=0x7fc54c0b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc54c0c9e40 0x7fc54c0b68f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc53c011690 tx=0x7fc53c011b50 comp rx=0 tx=0).stop 2024-04-25T01:11:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 shutdown_connections 2024-04-25T01:11:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fc5300678f0 0x7fc530069db0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc54c09ee50 0x7fc54c0b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc54c0c9e40 0x7fc54c0b68f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 --2- 172.21.15.33:0/2949525201 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fc54c0a0b60 0x7fc54c09e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 >> 172.21.15.33:0/2949525201 conn(0x7fc54c0d3db0 msgr2=0x7fc54c160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 shutdown_connections 2024-04-25T01:11:54.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:54.987+0000 7fc553266640 1 -- 172.21.15.33:0/2949525201 wait complete. 2024-04-25T01:11:54.996 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-04-25T01:11:53.163057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":229516,"kb_used_data":5024,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745204,"statfs":{"total":773094113280,"available":772859088896,"internally_reserved":0,"allocated":5144576,"data_stored":2911316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7677,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002362"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T01:11:52.781200+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:11:52.781200+0000","last_peered":"2024-04-25T01:11:52.781200+0000","last_clean":"2024-04-25T01:11:52.781200+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:11:52.781200+0000","last_undegraded":"2024-04-25T01:11:52.781200+0000","last_fullsized":"2024-04-25T01:11:52.781200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":814200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":212016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":217858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31668,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":499712,"data_stored":218504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":211524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":211546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1077248,"data_stored":801836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:11:54.997 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T01:11:54.997 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:11:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd611d6640 1 Processor -- start 2024-04-25T01:11:55.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd611d6640 1 -- start start 2024-04-25T01:11:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd611d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd611d6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbd5c0d48d0 0x7fbd5c0cef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd611d6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5c0d10b0 con 0x7fbd5c0d03e0 2024-04-25T01:11:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd611d6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fbd5c0d1de0 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd611d6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5c0d1390 con 0x7fbd5c15e470 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5b59e640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbd5c0d48d0 0x7fbd5c0cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c0ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:51462/0 (socket says 172.21.15.33:51462) 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5a59c640 1 -- 172.21.15.33:0/4117336528 learned_addr learned my addr 172.21.15.33:0/4117336528 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5ad9d640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fbd5c0d03e0 0x7fbd5c0d07c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:50752/0 (socket says 172.21.15.33:50752) 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5a59c640 1 -- 172.21.15.33:0/4117336528 >> v2:172.21.15.138:3300/0 conn(0x7fbd5c0d48d0 msgr2=0x7fbd5c0cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5a59c640 1 --2- 172.21.15.33:0/4117336528 >> v2:172.21.15.138:3300/0 conn(0x7fbd5c0d48d0 0x7fbd5c0cef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5a59c640 1 -- 172.21.15.33:0/4117336528 >> v1:172.21.15.138:6789/0 conn(0x7fbd5c0d03e0 legacy=0x7fbd5c0d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.135+0000 7fbd5a59c640 1 -- 172.21.15.33:0/4117336528 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c0d1670 con 0x7fbd5c15e470 2024-04-25T01:11:55.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd5a59c640 1 --2- 172.21.15.33:0/4117336528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c0ce9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009870 tx=0x7fbd4401c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dab55574032d1e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:55.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd59d9b640 1 -- 172.21.15.33:0/4117336528 <== mon.2 v2:172.21.15.138:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440b1070 con 0x7fbd5c15e470 2024-04-25T01:11:55.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd59d9b640 1 -- 172.21.15.33:0/4117336528 <== mon.2 v2:172.21.15.138:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd440a8050 con 0x7fbd5c15e470 2024-04-25T01:11:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd59d9b640 1 -- 172.21.15.33:0/4117336528 <== mon.2 v2:172.21.15.138:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440ac860 con 0x7fbd5c15e470 2024-04-25T01:11:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 -- 172.21.15.33:0/4117336528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 msgr2=0x7fbd5c0ce9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 --2- 172.21.15.33:0/4117336528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c0ce9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009870 tx=0x7fbd4401c4d0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 -- 172.21.15.33:0/4117336528 shutdown_connections 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 --2- 172.21.15.33:0/4117336528 >> v2:172.21.15.138:3300/0 conn(0x7fbd5c0d48d0 0x7fbd5c0cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 --2- 172.21.15.33:0/4117336528 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c0ce9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 -- 172.21.15.33:0/4117336528 >> 172.21.15.33:0/4117336528 conn(0x7fbd5c0d3db0 msgr2=0x7fbd5c0d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 -- 172.21.15.33:0/4117336528 shutdown_connections 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 -- 172.21.15.33:0/4117336528 wait complete. 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 Processor -- start 2024-04-25T01:11:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.139+0000 7fbd611d6640 1 -- start start 2024-04-25T01:11:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c0d03e0 0x7fbd5c09fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd5c0d48d0 0x7fbd5c09de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c16adc0 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c0cfe60 con 0x7fbd5c0d03e0 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5c16af40 con 0x7fbd5c15e470 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd5c0d48d0 0x7fbd5c09de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd5c0d48d0 0x7fbd5c09de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42808/0 (socket says 172.21.15.33:42808) 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 -- 172.21.15.33:0/3297637659 learned_addr learned my addr 172.21.15.33:0/3297637659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5b59e640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5ad9d640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c0d03e0 0x7fbd5c09fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 -- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 msgr2=0x7fbd5c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 -- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c0d03e0 msgr2=0x7fbd5c09fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c0d03e0 0x7fbd5c09fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 -- 172.21.15.33:0/3297637659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c145bb0 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5a59c640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd5c0d48d0 0x7fbd5c09de10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401ab80 tx=0x7fbd4401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:55.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd537fe640 1 -- 172.21.15.33:0/3297637659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440b1040 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd537fe640 1 -- 172.21.15.33:0/3297637659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd440bc070 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c145e50 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd537fe640 1 -- 172.21.15.33:0/3297637659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440b7890 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c146410 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c069040 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd537fe640 1 -- 172.21.15.33:0/3297637659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbd440b7a30 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd537fe640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd3c0678c0 0x7fbd3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd537fe640 1 -- 172.21.15.33:0/3297637659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fbd4411ff30 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.143+0000 7fbd5ad9d640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd3c0678c0 0x7fbd3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.147+0000 7fbd5ad9d640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd3c0678c0 0x7fbd3c069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c15de90 tx=0x7fbd4c007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:55.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.151+0000 7fbd537fe640 1 -- 172.21.15.33:0/3297637659 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fbd440c6020 con 0x7fbd5c0d48d0 2024-04-25T01:11:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.307+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd5c0d36f0 con 0x7fbd3c0678c0 2024-04-25T01:11:55.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.307+0000 7fbd537fe640 1 -- 172.21.15.33:0/3297637659 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13489 (secure 0 0 0) 0x7fbd5c0d36f0 con 0x7fbd3c0678c0 2024-04-25T01:11:55.300 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:55.300 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd3c0678c0 msgr2=0x7fbd3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd3c0678c0 0x7fbd3c069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c15de90 tx=0x7fbd4c007cf0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd5c0d48d0 msgr2=0x7fbd5c09de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd5c0d48d0 0x7fbd5c09de10 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401ab80 tx=0x7fbd4401ad10 comp rx=0 tx=0).stop 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 shutdown_connections 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fbd3c0678c0 0x7fbd3c069d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fbd5c15e470 0x7fbd5c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd5c0d48d0 0x7fbd5c09de10 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 --2- 172.21.15.33:0/3297637659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd5c0d03e0 0x7fbd5c09fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 >> 172.21.15.33:0/3297637659 conn(0x7fbd5c0d3db0 msgr2=0x7fbd5c0b6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:55.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 shutdown_connections 2024-04-25T01:11:55.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.311+0000 7fbd611d6640 1 -- 172.21.15.33:0/3297637659 wait complete. 2024-04-25T01:11:55.318 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-25T01:11:55.163579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":229516,"kb_used_data":5024,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745204,"statfs":{"total":773094113280,"available":772859088896,"internally_reserved":0,"allocated":5144576,"data_stored":2911316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002184"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T01:11:52.781200+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:11:52.781200+0000","last_peered":"2024-04-25T01:11:52.781200+0000","last_clean":"2024-04-25T01:11:52.781200+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:11:52.781200+0000","last_undegraded":"2024-04-25T01:11:52.781200+0000","last_fullsized":"2024-04-25T01:11:52.781200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509486,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":814200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":212016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27504,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344336,"statfs":{"total":96636764160,"available":96608600064,"internally_reserved":0,"allocated":495616,"data_stored":217858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509487,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31668,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340172,"statfs":{"total":96636764160,"available":96604336128,"internally_reserved":0,"allocated":499712,"data_stored":218504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":211524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":211546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32232,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339608,"statfs":{"total":96636764160,"available":96603758592,"internally_reserved":0,"allocated":1077248,"data_stored":801836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:11:55.318 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:11:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.451+0000 7f46f0a12640 1 Processor -- start 2024-04-25T01:11:55.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.451+0000 7f46f0a12640 1 -- start start 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46f0a12640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46ec145470 0x7f46ec15c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46f0a12640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0b5530 0x7f46ec0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46f0a12640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46ec15f9a0 con 0x7f46ec0cf0e0 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46f0a12640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f46ec15f750 con 0x7f46ec145470 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46f0a12640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec15fc80 con 0x7f46ec0b5530 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46ec145470 0x7f46ec15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ead9d640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0b5530 0x7f46ec0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46ec145470 0x7f46ec15c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:36450/0 (socket says 172.21.15.33:36450) 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46e9d9b640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f46ec0cf0e0 0x7f46ec0b4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.33:50760/0 (socket says 172.21.15.33:50760) 2024-04-25T01:11:55.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 -- 172.21.15.33:0/3311848207 learned_addr learned my addr 172.21.15.33:0/3311848207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 -- 172.21.15.33:0/3311848207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0b5530 msgr2=0x7f46ec0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 --2- 172.21.15.33:0/3311848207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0b5530 0x7f46ec0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 -- 172.21.15.33:0/3311848207 >> v1:172.21.15.138:6789/0 conn(0x7f46ec0cf0e0 legacy=0x7f46ec0b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 -- 172.21.15.33:0/3311848207 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec15ff60 con 0x7f46ec145470 2024-04-25T01:11:55.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.455+0000 7f46ea59c640 1 --2- 172.21.15.33:0/3311848207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec15c160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009870 tx=0x7f46dc01c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a283bb6fe300f09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46e959a640 1 -- 172.21.15.33:0/3311848207 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0a7070 con 0x7f46ec145470 2024-04-25T01:11:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46e959a640 1 -- 172.21.15.33:0/3311848207 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46dc0b0070 con 0x7f46ec145470 2024-04-25T01:11:55.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46e959a640 1 -- 172.21.15.33:0/3311848207 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0ab890 con 0x7f46ec145470 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- 172.21.15.33:0/3311848207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 msgr2=0x7f46ec15c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3311848207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec15c160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f46dc009870 tx=0x7f46dc01c560 comp rx=0 tx=0).stop 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- 172.21.15.33:0/3311848207 shutdown_connections 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3311848207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0b5530 0x7f46ec0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3311848207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec15c160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- 172.21.15.33:0/3311848207 >> 172.21.15.33:0/3311848207 conn(0x7f46ec0b1810 msgr2=0x7f46ec0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- 172.21.15.33:0/3311848207 shutdown_connections 2024-04-25T01:11:55.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- 172.21.15.33:0/3311848207 wait complete. 2024-04-25T01:11:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 Processor -- start 2024-04-25T01:11:55.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- start start 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ec0b5530 0x7f46ec0ae910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0cf0e0 0x7f46ec13a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec13a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec168c70 con 0x7f46ec0b5530 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec168af0 con 0x7f46ec145470 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46f0a12640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec168df0 con 0x7f46ec0cf0e0 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ec0b5530 0x7f46ec0ae910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46ea59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ec0b5530 0x7f46ec0ae910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42814/0 (socket says 172.21.15.33:42814) 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.459+0000 7f46ea59c640 1 -- 172.21.15.33:0/3100996958 learned_addr learned my addr 172.21.15.33:0/3100996958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e9d9b640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0cf0e0 0x7f46ec13a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ead9d640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec13a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ea59c640 1 -- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0cf0e0 msgr2=0x7f46ec13a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ea59c640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0cf0e0 0x7f46ec13a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ea59c640 1 -- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 msgr2=0x7f46ec13a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ea59c640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec13a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ea59c640 1 -- 172.21.15.33:0/3100996958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec0aee50 con 0x7f46ec0b5530 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e9d9b640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0cf0e0 0x7f46ec13a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ead9d640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec13a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46ea59c640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ec0b5530 0x7f46ec0ae910 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f46dc01ce00 tx=0x7f46dc01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e2ffd640 1 -- 172.21.15.33:0/3100996958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0be5f0 con 0x7f46ec0b5530 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e2ffd640 1 -- 172.21.15.33:0/3100996958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46dc0be790 con 0x7f46ec0b5530 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec13f2c0 con 0x7f46ec0b5530 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e2ffd640 1 -- 172.21.15.33:0/3100996958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46dc0beb80 con 0x7f46ec0b5530 2024-04-25T01:11:55.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec13f880 con 0x7f46ec0b5530 2024-04-25T01:11:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec069040 con 0x7f46ec0b5530 2024-04-25T01:11:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e2ffd640 1 -- 172.21.15.33:0/3100996958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46dc0b43f0 con 0x7f46ec0b5530 2024-04-25T01:11:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e2ffd640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46cc0678c0 0x7f46cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:55.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e2ffd640 1 -- 172.21.15.33:0/3100996958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f46dc120440 con 0x7f46ec0b5530 2024-04-25T01:11:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e9d9b640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46cc0678c0 0x7f46cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:55.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.463+0000 7f46e9d9b640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46cc0678c0 0x7f46cc069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f46d4007fd0 tx=0x7f46d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:55.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.467+0000 7f46e2ffd640 1 -- 172.21.15.33:0/3100996958 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f46dc0c4060 con 0x7f46ec0b5530 2024-04-25T01:11:55.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.631+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46ec071420 con 0x7f46ec0b5530 2024-04-25T01:11:55.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46e2ffd640 1 -- 172.21.15.33:0/3100996958 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13281 (secure 0 0 0) 0x7f46dc016150 con 0x7f46ec0b5530 2024-04-25T01:11:55.625 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:11:55.625 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":44,"fsid":"10df718f-e6ff-4eb1-9854-6e8a751898ff","created":"2024-04-25T01:08:06.431795+0000","modified":"2024-04-25T01:11:52.589968+0000","last_up_change":"2024-04-25T01:08:17.160602+0000","last_in_change":"2024-04-25T01:08:08.803308+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:08:15.214297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a55c2d3b-5b10-4106-a9b3-2041d0435aa0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6825","nonce":1142279268}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6827","nonce":1142279268}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6831","nonce":1142279268}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":1142279268},{"type":"v1","addr":"172.21.15.33:6829","nonce":1142279268}]},"public_addr":"172.21.15.33:6825/1142279268","cluster_addr":"172.21.15.33:6827/1142279268","heartbeat_back_addr":"172.21.15.33:6831/1142279268","heartbeat_front_addr":"172.21.15.33:6829/1142279268","state":["exists","up"]},{"osd":1,"uuid":"b941572f-acd0-4d94-af10-64f4735614a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6809","nonce":1105804411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6811","nonce":1105804411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6815","nonce":1105804411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":1105804411},{"type":"v1","addr":"172.21.15.33:6813","nonce":1105804411}]},"public_addr":"172.21.15.33:6809/1105804411","cluster_addr":"172.21.15.33:6811/1105804411","heartbeat_back_addr":"172.21.15.33:6815/1105804411","heartbeat_front_addr":"172.21.15.33:6813/1105804411","state":["exists","up"]},{"osd":2,"uuid":"36aa7662-65b1-413e-a94f-099070af8a2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6801","nonce":1000748225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6803","nonce":1000748225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6807","nonce":1000748225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":1000748225},{"type":"v1","addr":"172.21.15.33:6805","nonce":1000748225}]},"public_addr":"172.21.15.33:6801/1000748225","cluster_addr":"172.21.15.33:6803/1000748225","heartbeat_back_addr":"172.21.15.33:6807/1000748225","heartbeat_front_addr":"172.21.15.33:6805/1000748225","state":["exists","up"]},{"osd":3,"uuid":"3d561dd1-ed88-488d-87b6-f3017b4dc353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6817","nonce":2195826364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6819","nonce":2195826364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6823","nonce":2195826364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":2195826364},{"type":"v1","addr":"172.21.15.33:6821","nonce":2195826364}]},"public_addr":"172.21.15.33:6817/2195826364","cluster_addr":"172.21.15.33:6819/2195826364","heartbeat_back_addr":"172.21.15.33:6823/2195826364","heartbeat_front_addr":"172.21.15.33:6821/2195826364","state":["exists","up"]},{"osd":4,"uuid":"ff949513-c8e5-489b-8992-5b673f1e1cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6817","nonce":100119801}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6819","nonce":100119801}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6823","nonce":100119801}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":100119801},{"type":"v1","addr":"172.21.15.138:6821","nonce":100119801}]},"public_addr":"172.21.15.138:6817/100119801","cluster_addr":"172.21.15.138:6819/100119801","heartbeat_back_addr":"172.21.15.138:6823/100119801","heartbeat_front_addr":"172.21.15.138:6821/100119801","state":["exists","up"]},{"osd":5,"uuid":"2512be72-afd8-4caf-b076-9c0a07dbac06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6801","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6804","nonce":1955200876}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6808","nonce":1955200876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6815","nonce":1955200876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1955200876},{"type":"v1","addr":"172.21.15.138:6812","nonce":1955200876}]},"public_addr":"172.21.15.138:6804/1955200876","cluster_addr":"172.21.15.138:6808/1955200876","heartbeat_back_addr":"172.21.15.138:6815/1955200876","heartbeat_front_addr":"172.21.15.138:6812/1955200876","state":["exists","up"]},{"osd":6,"uuid":"44142ad5-e804-4165-bcb2-c266a808a644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6825","nonce":1315598862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6827","nonce":1315598862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6831","nonce":1315598862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1315598862},{"type":"v1","addr":"172.21.15.138:6829","nonce":1315598862}]},"public_addr":"172.21.15.138:6825/1315598862","cluster_addr":"172.21.15.138:6827/1315598862","heartbeat_back_addr":"172.21.15.138:6831/1315598862","heartbeat_front_addr":"172.21.15.138:6829/1315598862","state":["exists","up"]},{"osd":7,"uuid":"7cdb7435-2684-4dec-9321-53d1cb275222","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6802","nonce":400031455}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6803","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6805","nonce":400031455}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6811","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6813","nonce":400031455}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":400031455},{"type":"v1","addr":"172.21.15.138:6809","nonce":400031455}]},"public_addr":"172.21.15.138:6802/400031455","cluster_addr":"172.21.15.138:6805/400031455","heartbeat_back_addr":"172.21.15.138:6813/400031455","heartbeat_front_addr":"172.21.15.138:6809/400031455","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:13.834701+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.439034+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.298567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.392400+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.120773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.027872+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.136918+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:08:14.114722+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6834/3726672082":"2024-04-26T01:11:49.577447+0000","172.21.15.33:6832/3726672082":"2024-04-26T01:11:49.577447+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46cc0678c0 msgr2=0x7f46cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46cc0678c0 0x7f46cc069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f46d4007fd0 tx=0x7f46d4009040 comp rx=0 tx=0).stop 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ec0b5530 msgr2=0x7f46ec0ae910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ec0b5530 0x7f46ec0ae910 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f46dc01ce00 tx=0x7f46dc01ada0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 shutdown_connections 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f46cc0678c0 0x7f46cc069d80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46ec145470 0x7f46ec13a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f46ec0cf0e0 0x7f46ec13a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 --2- 172.21.15.33:0/3100996958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46ec0b5530 0x7f46ec0ae910 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 >> 172.21.15.33:0/3100996958 conn(0x7f46ec0b1810 msgr2=0x7f46ec0b0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 shutdown_connections 2024-04-25T01:11:55.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:55.635+0000 7f46f0a12640 1 -- 172.21.15.33:0/3100996958 wait complete. 2024-04-25T01:11:56.646 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T01:11:56.646 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:11:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f543d4640 1 Processor -- start 2024-04-25T01:11:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f543d4640 1 -- start start 2024-04-25T01:11:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f543d4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f4c144700 0x7f6f4c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f543d4640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6f4c06ca40 0x7f6f4c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f543d4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f4c15efa0 con 0x7f6f4c072cf0 2024-04-25T01:11:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f543d4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4c072040 con 0x7f6f4c144700 2024-04-25T01:11:56.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f543d4640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6f4c15f280 con 0x7f6f4c06ca40 2024-04-25T01:11:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f52170640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6f4c144700 0x7f6f4c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f52971640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6f4c072cf0 0x7f6f4c0cfc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33164/0 (socket says 172.21.15.33:33164) 2024-04-25T01:11:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f52971640 1 -- 172.21.15.33:0/1479704448 learned_addr learned my addr 172.21.15.33:0/1479704448 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f5196f640 1 --2- 172.21.15.33:0/1479704448 >> v2:172.21.15.138:3300/0 conn(0x7f6f4c06ca40 0x7f6f4c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f52170640 1 -- 172.21.15.33:0/1479704448 >> v2:172.21.15.138:3300/0 conn(0x7f6f4c06ca40 msgr2=0x7f6f4c0727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:56.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f52170640 1 --2- 172.21.15.33:0/1479704448 >> v2:172.21.15.138:3300/0 conn(0x7f6f4c06ca40 0x7f6f4c0727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f52170640 1 -- 172.21.15.33:0/1479704448 >> v1:172.21.15.33:6789/0 conn(0x7f6f4c072cf0 legacy=0x7f6f4c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f52170640 1 -- 172.21.15.33:0/1479704448 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c15f560 con 0x7f6f4c144700 2024-04-25T01:11:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f5116e640 1 -- 172.21.15.33:0/1479704448 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1933395105 0 0) 0x7f6f4c15efa0 con 0x7f6f4c072cf0 2024-04-25T01:11:56.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.795+0000 7f6f5196f640 1 --2- 172.21.15.33:0/1479704448 >> v2:172.21.15.138:3300/0 conn(0x7f6f4c06ca40 0x7f6f4c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f52170640 1 --2- 172.21.15.33:0/1479704448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c144700 0x7f6f4c15e770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0b1730 tx=0x7f6f4801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f1ae80122a5138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f5116e640 1 -- 172.21.15.33:0/1479704448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f480ab430 con 0x7f6f4c144700 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f5116e640 1 -- 172.21.15.33:0/1479704448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f480b0070 con 0x7f6f4c144700 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f5116e640 1 -- 172.21.15.33:0/1479704448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f480ab7f0 con 0x7f6f4c144700 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 -- 172.21.15.33:0/1479704448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c144700 msgr2=0x7f6f4c15e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 --2- 172.21.15.33:0/1479704448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c144700 0x7f6f4c15e770 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c0b1730 tx=0x7f6f4801c450 comp rx=0 tx=0).stop 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 -- 172.21.15.33:0/1479704448 shutdown_connections 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 --2- 172.21.15.33:0/1479704448 >> v2:172.21.15.138:3300/0 conn(0x7f6f4c06ca40 0x7f6f4c0727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 --2- 172.21.15.33:0/1479704448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c144700 0x7f6f4c15e770 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 -- 172.21.15.33:0/1479704448 >> 172.21.15.33:0/1479704448 conn(0x7f6f4c06b3e0 msgr2=0x7f6f4c06b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 -- 172.21.15.33:0/1479704448 shutdown_connections 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 -- 172.21.15.33:0/1479704448 wait complete. 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.799+0000 7f6f543d4640 1 Processor -- start 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 -- start start 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c06ca40 0x7f6f4c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f4c072cf0 0x7f6f4c1437d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f4c144700 0x7f6f4c0b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0d0320 con 0x7f6f4c06ca40 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4c0cd860 con 0x7f6f4c072cf0 2024-04-25T01:11:56.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4c0d04a0 con 0x7f6f4c144700 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c06ca40 0x7f6f4c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c06ca40 0x7f6f4c159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42822/0 (socket says 172.21.15.33:42822) 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 -- 172.21.15.33:0/3448835725 learned_addr learned my addr 172.21.15.33:0/3448835725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52971640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f4c144700 0x7f6f4c0b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f5196f640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f4c072cf0 0x7f6f4c1437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 -- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f4c144700 msgr2=0x7f6f4c0b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6f4c144700 0x7f6f4c0b2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 -- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f4c072cf0 msgr2=0x7f6f4c1437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f4c072cf0 0x7f6f4c1437d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 -- 172.21.15.33:0/3448835725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4c13c420 con 0x7f6f4c06ca40 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f52170640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c06ca40 0x7f6f4c159820 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6f48019060 tx=0x7f6f4801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48004220 con 0x7f6f4c06ca40 2024-04-25T01:11:56.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f480043c0 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f4c15d460 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f48003820 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4c15da20 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f4c15d2b0 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6f480039c0 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f42ffd640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f300678c0 0x7f6f30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.803+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f6f480a8020 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f42ffd640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f6f3006dd60 0x7f6f300701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 --> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f48003e50 con 0x7f6f3006dd60 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6f48127050 con 0x7f6f4c06ca40 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f52971640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f6f3006dd60 0x7f6f300701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f5196f640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f300678c0 0x7f6f30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f52971640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f6f3006dd60 0x7f6f300701a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f5196f640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f300678c0 0x7f6f30069d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006fd0 tx=0x7f6f3c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:56.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.807+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== osd.0 v2:172.21.15.33:6824/1142279268 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f48003e50 con 0x7f6f3006dd60 2024-04-25T01:11:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 --> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6f4c069040 con 0x7f6f3006dd60 2024-04-25T01:11:56.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f42ffd640 1 -- 172.21.15.33:0/3448835725 <== osd.0 v2:172.21.15.33:6824/1142279268 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6f4c069040 con 0x7f6f3006dd60 2024-04-25T01:11:56.810 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:11:56.811 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:11:56.811 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f6f3006dd60 msgr2=0x7f6f300701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:6824/1142279268,v1:172.21.15.33:6825/1142279268] conn(0x7f6f3006dd60 0x7f6f300701a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f300678c0 msgr2=0x7f6f30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f300678c0 0x7f6f30069d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c006fd0 tx=0x7f6f3c008040 comp rx=0 tx=0).stop 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c06ca40 msgr2=0x7f6f4c159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 --2- 172.21.15.33:0/3448835725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6f4c06ca40 0x7f6f4c159820 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6f48019060 tx=0x7f6f4801ac90 comp rx=0 tx=0).stop 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f52971640 1 -- 172.21.15.33:0/3448835725 reap_dead start 2024-04-25T01:11:56.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 shutdown_connections 2024-04-25T01:11:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 >> 172.21.15.33:0/3448835725 conn(0x7f6f4c06b3e0 msgr2=0x7f6f4c06b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 shutdown_connections 2024-04-25T01:11:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.819+0000 7f6f543d4640 1 -- 172.21.15.33:0/3448835725 wait complete. 2024-04-25T01:11:56.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T01:11:56.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.947+0000 7f9fded14640 1 Processor -- start 2024-04-25T01:11:56.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.947+0000 7f9fded14640 1 -- start start 2024-04-25T01:11:56.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.947+0000 7f9fded14640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd80d03e0 0x7f9fd80d07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.947+0000 7f9fded14640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9fd815e470 0x7f9fd80d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.947+0000 7f9fded14640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fd80d10b0 con 0x7f9fd80d48d0 2024-04-25T01:11:56.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.947+0000 7f9fded14640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f9fd80d1de0 con 0x7f9fd815e470 2024-04-25T01:11:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.947+0000 7f9fded14640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd80d1390 con 0x7f9fd80d03e0 2024-04-25T01:11:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fdcab0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd80d03e0 0x7f9fd80d07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd7fff640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9fd815e470 0x7f9fd80d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fdcab0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd80d03e0 0x7f9fd80d07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3301/0 says I am v2:172.21.15.33:51518/0 (socket says 172.21.15.33:51518) 2024-04-25T01:11:56.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fdcab0640 1 -- 172.21.15.33:0/1373810657 learned_addr learned my addr 172.21.15.33:0/1373810657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:56.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd7fff640 1 -- 172.21.15.33:0/1373810657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd80d03e0 msgr2=0x7f9fd80d07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:56.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd7fff640 1 --2- 172.21.15.33:0/1373810657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd80d03e0 0x7f9fd80d07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd7fff640 1 -- 172.21.15.33:0/1373810657 >> v1:172.21.15.138:6789/0 conn(0x7f9fd80d48d0 legacy=0x7f9fd80d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd7fff640 1 -- 172.21.15.33:0/1373810657 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd80d1670 con 0x7f9fd815e470 2024-04-25T01:11:56.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd7fff640 1 --2- 172.21.15.33:0/1373810657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd815e470 0x7f9fd80d2270 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0097c0 tx=0x7f9fd80d36d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba26650e74d26dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd77fe640 1 -- 172.21.15.33:0/1373810657 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0b0070 con 0x7f9fd815e470 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd77fe640 1 -- 172.21.15.33:0/1373810657 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0ab600 con 0x7f9fd815e470 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fd77fe640 1 -- 172.21.15.33:0/1373810657 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0ab9d0 con 0x7f9fd815e470 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 -- 172.21.15.33:0/1373810657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd815e470 msgr2=0x7f9fd80d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 --2- 172.21.15.33:0/1373810657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd815e470 0x7f9fd80d2270 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0097c0 tx=0x7f9fd80d36d0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 -- 172.21.15.33:0/1373810657 shutdown_connections 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 --2- 172.21.15.33:0/1373810657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd815e470 0x7f9fd80d2270 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 --2- 172.21.15.33:0/1373810657 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd80d03e0 0x7f9fd80d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 -- 172.21.15.33:0/1373810657 >> 172.21.15.33:0/1373810657 conn(0x7f9fd80d3db0 msgr2=0x7f9fd80d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 -- 172.21.15.33:0/1373810657 shutdown_connections 2024-04-25T01:11:56.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.951+0000 7f9fded14640 1 -- 172.21.15.33:0/1373810657 wait complete. 2024-04-25T01:11:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 Processor -- start 2024-04-25T01:11:56.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 -- start start 2024-04-25T01:11:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd80d03e0 0x7f9fd80cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd80d48d0 0x7f9fd80b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd815e470 0x7f9fd80cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd816af40 con 0x7f9fd80d48d0 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd816adc0 con 0x7f9fd80d03e0 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd816b0c0 con 0x7f9fd815e470 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd80d48d0 0x7f9fd80b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd80d48d0 0x7f9fd80b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42832/0 (socket says 172.21.15.33:42832) 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 -- 172.21.15.33:0/3926068322 learned_addr learned my addr 172.21.15.33:0/3926068322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fdcab0640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd80d03e0 0x7f9fd80cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fdd2b1640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd815e470 0x7f9fd80cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 -- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd815e470 msgr2=0x7f9fd80cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd815e470 0x7f9fd80cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 -- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd80d03e0 msgr2=0x7f9fd80cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:56.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd80d03e0 0x7f9fd80cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 -- 172.21.15.33:0/3926068322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd8154b00 con 0x7f9fd80d48d0 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd7fff640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd80d48d0 0x7f9fd80b6980 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc019f80 tx=0x7f9fcc0190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd57fa640 1 -- 172.21.15.33:0/3926068322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0b0040 con 0x7f9fd80d48d0 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd57fa640 1 -- 172.21.15.33:0/3926068322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0ba070 con 0x7f9fd80d48d0 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd80b6fb0 con 0x7f9fd80d48d0 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fd57fa640 1 -- 172.21.15.33:0/3926068322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0b4630 con 0x7f9fd80d48d0 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd8155080 con 0x7f9fd80d48d0 2024-04-25T01:11:56.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.955+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd8069040 con 0x7f9fd80d48d0 2024-04-25T01:11:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.959+0000 7f9fd57fa640 1 -- 172.21.15.33:0/3926068322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9fcc0b47d0 con 0x7f9fd80d48d0 2024-04-25T01:11:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.959+0000 7f9fd57fa640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9fc40678f0 0x7f9fc4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.959+0000 7f9fd57fa640 1 -- 172.21.15.33:0/3926068322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f9fcc11f6d0 con 0x7f9fd80d48d0 2024-04-25T01:11:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.959+0000 7f9fdcab0640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9fc40678f0 0x7f9fc4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:56.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.959+0000 7f9fdcab0640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9fc40678f0 0x7f9fc4069db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009ae0 tx=0x7f9fc8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:56.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:56.963+0000 7f9fd57fa640 1 -- 172.21.15.33:0/3926068322 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9fcc016090 con 0x7f9fd80d48d0 2024-04-25T01:11:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.123+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f9fd80d4330 con 0x7f9fc40678f0 2024-04-25T01:11:57.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.123+0000 7f9fd57fa640 1 -- 172.21.15.33:0/3926068322 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9fd80d4330 con 0x7f9fc40678f0 2024-04-25T01:11:57.115 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9fc40678f0 msgr2=0x7f9fc4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9fc40678f0 0x7f9fc4069db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009ae0 tx=0x7f9fc8008040 comp rx=0 tx=0).stop 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd80d48d0 msgr2=0x7f9fd80b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd80d48d0 0x7f9fd80b6980 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc019f80 tx=0x7f9fcc0190f0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 shutdown_connections 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f9fc40678f0 0x7f9fc4069db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f9fd815e470 0x7f9fd80cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fd80d48d0 0x7f9fd80b6980 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 --2- 172.21.15.33:0/3926068322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fd80d03e0 0x7f9fd80cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 >> 172.21.15.33:0/3926068322 conn(0x7f9fd80d3db0 msgr2=0x7f9fd80c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 shutdown_connections 2024-04-25T01:11:57.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.127+0000 7f9fded14640 1 -- 172.21.15.33:0/3926068322 wait complete. 2024-04-25T01:11:57.134 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T01:11:57.134 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:11:57.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f587642b640 1 Processor -- start 2024-04-25T01:11:57.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f587642b640 1 -- start start 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f587642b640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5870072500 0x7f587015b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f587642b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587015bf10 con 0x7f587015b790 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f587642b640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f587006ded0 con 0x7f58700b5500 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f587642b640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f587015c1f0 con 0x7f5870072500 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f587015b790 0x7f587006e9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33174/0 (socket says 172.21.15.33:33174) 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 -- 172.21.15.33:0/777245529 learned_addr learned my addr 172.21.15.33:0/777245529 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586f7fe640 1 --2- 172.21.15.33:0/777245529 >> v2:172.21.15.138:3300/0 conn(0x7f5870072500 0x7f587015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740395441 0 0) 0x7f587015bf10 con 0x7f587015b790 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5858003530 con 0x7f587015b790 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1480974568 0 0) 0x7f5858003530 con 0x7f587015b790 2024-04-25T01:11:57.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 >> v2:172.21.15.138:3300/0 conn(0x7f5870072500 msgr2=0x7f587015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 --2- 172.21.15.33:0/777245529 >> v2:172.21.15.138:3300/0 conn(0x7f5870072500 0x7f587015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 >> v1:172.21.15.138:6789/0 conn(0x7f58700b5500 legacy=0x7f5870071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587006e1b0 con 0x7f587015b790 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f5868001a00 con 0x7f587015b790 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 >> v1:172.21.15.33:6789/0 conn(0x7f587015b790 legacy=0x7f587006e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580039c0 0x7f5858003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5858004810 0x7f5858004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58580052f0 0x7f585800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f587015c1f0 con 0x7f58580052f0 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f585800e3d0 con 0x7f58580039c0 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f585800e5d0 con 0x7f5858004810 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58580052f0 0x7f585800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586f7fe640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5858004810 0x7f5858004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586ffff640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580039c0 0x7f5858003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 -- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5858004810 msgr2=0x7f5858004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5858004810 0x7f5858004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 -- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580039c0 msgr2=0x7f5858003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580039c0 0x7f5858003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f58749c8640 1 -- 172.21.15.33:0/777245529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5858003530 con 0x7f58580052f0 2024-04-25T01:11:57.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.267+0000 7f586ffff640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580039c0 0x7f5858003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:57.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f58749c8640 1 -- 172.21.15.33:0/777245529 reap_dead start 2024-04-25T01:11:57.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f58749c8640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58580052f0 0x7f585800dbe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5868002240 tx=0x7f586801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b20303c4fff51a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586800a4a0 con 0x7f58580052f0 2024-04-25T01:11:57.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58680ad650 con 0x7f58580052f0 2024-04-25T01:11:57.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f586effd640 1 -- 172.21.15.33:0/777245529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58680ada40 con 0x7f58580052f0 2024-04-25T01:11:57.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 -- 172.21.15.33:0/777245529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58580052f0 msgr2=0x7f585800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58580052f0 0x7f585800dbe0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5868002240 tx=0x7f586801d360 comp rx=0 tx=0).stop 2024-04-25T01:11:57.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 -- 172.21.15.33:0/777245529 shutdown_connections 2024-04-25T01:11:57.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 --2- 172.21.15.33:0/777245529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58580052f0 0x7f585800dbe0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 -- 172.21.15.33:0/777245529 >> 172.21.15.33:0/777245529 conn(0x7f58700b2720 msgr2=0x7f58700b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 -- 172.21.15.33:0/777245529 shutdown_connections 2024-04-25T01:11:57.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 -- 172.21.15.33:0/777245529 wait complete. 2024-04-25T01:11:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 Processor -- start 2024-04-25T01:11:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.271+0000 7f587642b640 1 -- start start 2024-04-25T01:11:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580052f0 0x7f58700cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5870148fd0 0x7f587014bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5870149ee0 0x7f587014a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f587015b5e0 con 0x7f5870148fd0 2024-04-25T01:11:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f587015b460 con 0x7f58580052f0 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f587006f8d0 con 0x7f5870149ee0 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5870148fd0 0x7f587014bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5870148fd0 0x7f587014bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42854/0 (socket says 172.21.15.33:42854) 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 -- 172.21.15.33:0/1784692294 learned_addr learned my addr 172.21.15.33:0/1784692294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586ffff640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580052f0 0x7f58700cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f58749c8640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5870149ee0 0x7f587014a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 -- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5870149ee0 msgr2=0x7f587014a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f5870149ee0 0x7f587014a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 -- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580052f0 msgr2=0x7f58700cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58580052f0 0x7f58700cff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 -- 172.21.15.33:0/1784692294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587014d230 con 0x7f5870148fd0 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586f7fe640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5870148fd0 0x7f587014bdb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f58640115c0 tx=0x7f5864011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864015870 con 0x7f5870148fd0 2024-04-25T01:11:57.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5864015a10 con 0x7f5870148fd0 2024-04-25T01:11:57.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587014c620 con 0x7f5870148fd0 2024-04-25T01:11:57.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864015dc0 con 0x7f5870148fd0 2024-04-25T01:11:57.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587014cbe0 con 0x7f5870148fd0 2024-04-25T01:11:57.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5870069040 con 0x7f5870148fd0 2024-04-25T01:11:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5864013400 con 0x7f5870148fd0 2024-04-25T01:11:57.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f585c0678c0 0x7f585c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f5864086460 con 0x7f5870148fd0 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7f585c06dd60 0x7f585c0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 --> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f586400ce90 con 0x7f585c06dd60 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f58640868f0 con 0x7f5870148fd0 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f586ffff640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f585c0678c0 0x7f585c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.275+0000 7f58749c8640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7f585c06dd60 0x7f585c0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.279+0000 7f58749c8640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7f585c06dd60 0x7f585c0701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.279+0000 7f586ffff640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f585c0678c0 0x7f585c069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5870153910 tx=0x7f5860009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.279+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== osd.1 v2:172.21.15.33:6808/1105804411 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f586400ce90 con 0x7f585c06dd60 2024-04-25T01:11:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 --> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5870072c10 con 0x7f585c06dd60 2024-04-25T01:11:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f586cff9640 1 -- 172.21.15.33:0/1784692294 <== osd.1 v2:172.21.15.33:6808/1105804411 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5870072c10 con 0x7f585c06dd60 2024-04-25T01:11:57.283 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:11:57.283 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:11:57.283 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7f585c06dd60 msgr2=0x7f585c0701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:6808/1105804411,v1:172.21.15.33:6809/1105804411] conn(0x7f585c06dd60 0x7f585c0701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f585c0678c0 msgr2=0x7f585c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f585c0678c0 0x7f585c069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5870153910 tx=0x7f5860009040 comp rx=0 tx=0).stop 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5870148fd0 msgr2=0x7f587014bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 --2- 172.21.15.33:0/1784692294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5870148fd0 0x7f587014bdb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f58640115c0 tx=0x7f5864011a80 comp rx=0 tx=0).stop 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f58749c8640 1 -- 172.21.15.33:0/1784692294 reap_dead start 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 shutdown_connections 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 >> 172.21.15.33:0/1784692294 conn(0x7f58700b2720 msgr2=0x7f58700ccd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 shutdown_connections 2024-04-25T01:11:57.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.291+0000 7f587642b640 1 -- 172.21.15.33:0/1784692294 wait complete. 2024-04-25T01:11:57.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T01:11:57.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 Processor -- start 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 -- start start 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a7c15e580 0x7f1a7c0b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1a7c0d48c0 0x7f1a7c0d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c0d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7c0d1720 con 0x7f1a7c15e580 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f1a7c0d1440 con 0x7f1a7c0d48c0 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a81792640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7c0b4240 con 0x7f1a7c0d0cb0 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a7c15e580 0x7f1a7c0b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1a7c15e580 0x7f1a7c0b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42862/0 (socket says 172.21.15.33:42862) 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 -- 172.21.15.33:0/3213782570 learned_addr learned my addr 172.21.15.33:0/3213782570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7b7fe640 1 --2- 172.21.15.33:0/3213782570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c0d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7a7fc640 1 --2- 172.21.15.33:0/3213782570 >> v2:172.21.15.138:3300/0 conn(0x7f1a7c0d48c0 0x7f1a7c0d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 -- 172.21.15.33:0/3213782570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 msgr2=0x7f1a7c0d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 --2- 172.21.15.33:0/3213782570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c0d1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 -- 172.21.15.33:0/3213782570 >> v2:172.21.15.138:3300/0 conn(0x7f1a7c0d48c0 msgr2=0x7f1a7c0d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 --2- 172.21.15.33:0/3213782570 >> v2:172.21.15.138:3300/0 conn(0x7f1a7c0d48c0 0x7f1a7c0d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.419+0000 7f1a7affd640 1 -- 172.21.15.33:0/3213782570 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c0b4520 con 0x7f1a7c15e580 2024-04-25T01:11:57.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a7affd640 1 --2- 172.21.15.33:0/3213782570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0b4b30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0c7800 tx=0x7f1a7001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fcc4a5e9b534f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a79ffb640 1 -- 172.21.15.33:0/3213782570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a700b0070 con 0x7f1a7c15e580 2024-04-25T01:11:57.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a79ffb640 1 -- 172.21.15.33:0/3213782570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a700ab600 con 0x7f1a7c15e580 2024-04-25T01:11:57.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a79ffb640 1 -- 172.21.15.33:0/3213782570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a700ab9d0 con 0x7f1a7c15e580 2024-04-25T01:11:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 -- 172.21.15.33:0/3213782570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 msgr2=0x7f1a7c0b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 --2- 172.21.15.33:0/3213782570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0b4b30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0c7800 tx=0x7f1a7001ccc0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 -- 172.21.15.33:0/3213782570 shutdown_connections 2024-04-25T01:11:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 --2- 172.21.15.33:0/3213782570 >> v2:172.21.15.138:3300/0 conn(0x7f1a7c0d48c0 0x7f1a7c0d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 --2- 172.21.15.33:0/3213782570 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c0d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 --2- 172.21.15.33:0/3213782570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0b4b30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 -- 172.21.15.33:0/3213782570 >> 172.21.15.33:0/3213782570 conn(0x7f1a7c0d3da0 msgr2=0x7f1a7c0d41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 -- 172.21.15.33:0/3213782570 shutdown_connections 2024-04-25T01:11:57.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 -- 172.21.15.33:0/3213782570 wait complete. 2024-04-25T01:11:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 Processor -- start 2024-04-25T01:11:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.423+0000 7f1a81792640 1 -- start start 2024-04-25T01:11:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a7c0d48c0 0x7f1a7c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c0ceff0 con 0x7f1a7c15e580 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7c0cee70 con 0x7f1a7c0d48c0 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1a7c0cf170 con 0x7f1a7c0d0cb0 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0cd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42864/0 (socket says 172.21.15.33:42864) 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 -- 172.21.15.33:0/2436554222 learned_addr learned my addr 172.21.15.33:0/2436554222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7affd640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7a7fc640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a7c0d48c0 0x7f1a7c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 -- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 msgr2=0x7f1a7c153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 -- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a7c0d48c0 msgr2=0x7f1a7c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a7c0d48c0 0x7f1a7c0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 -- 172.21.15.33:0/2436554222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7c152bf0 con 0x7f1a7c15e580 2024-04-25T01:11:57.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7b7fe640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0cd710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1a74007e60 tx=0x7f1a7400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a637fe640 1 -- 172.21.15.33:0/2436554222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74014620 con 0x7f1a7c15e580 2024-04-25T01:11:57.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a637fe640 1 -- 172.21.15.33:0/2436554222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a74019070 con 0x7f1a7c15e580 2024-04-25T01:11:57.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a637fe640 1 -- 172.21.15.33:0/2436554222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74014a20 con 0x7f1a7c15e580 2024-04-25T01:11:57.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7c152ed0 con 0x7f1a7c15e580 2024-04-25T01:11:57.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a7c153490 con 0x7f1a7c15e580 2024-04-25T01:11:57.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a7c069040 con 0x7f1a7c15e580 2024-04-25T01:11:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a637fe640 1 -- 172.21.15.33:0/2436554222 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1a74014bc0 con 0x7f1a7c15e580 2024-04-25T01:11:57.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a637fe640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a680678c0 0x7f1a68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a637fe640 1 -- 172.21.15.33:0/2436554222 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f1a740862a0 con 0x7f1a7c15e580 2024-04-25T01:11:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.427+0000 7f1a7affd640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a680678c0 0x7f1a68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.431+0000 7f1a7affd640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a680678c0 0x7f1a68069d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a700ad000 tx=0x7f1a70004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.431+0000 7f1a637fe640 1 -- 172.21.15.33:0/2436554222 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f1a74056fe0 con 0x7f1a7c15e580 2024-04-25T01:11:57.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.595+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f1a7c0cdc50 con 0x7f1a680678c0 2024-04-25T01:11:57.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.595+0000 7f1a637fe640 1 -- 172.21.15.33:0/2436554222 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1a7c0cdc50 con 0x7f1a680678c0 2024-04-25T01:11:57.588 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a680678c0 msgr2=0x7f1a68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a680678c0 0x7f1a68069d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f1a700ad000 tx=0x7f1a70004740 comp rx=0 tx=0).stop 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 msgr2=0x7f1a7c0cd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0cd710 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1a74007e60 tx=0x7f1a7400e4b0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 shutdown_connections 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1a680678c0 0x7f1a68069d80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a7c15e580 0x7f1a7c0cd710 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a7c0d48c0 0x7f1a7c0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 --2- 172.21.15.33:0/2436554222 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1a7c0d0cb0 0x7f1a7c153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 >> 172.21.15.33:0/2436554222 conn(0x7f1a7c0d3da0 msgr2=0x7f1a7c155100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 shutdown_connections 2024-04-25T01:11:57.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.599+0000 7f1a81792640 1 -- 172.21.15.33:0/2436554222 wait complete. 2024-04-25T01:11:57.608 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T01:11:57.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:11:57.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05f516b640 1 Processor -- start 2024-04-25T01:11:57.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05f516b640 1 -- start start 2024-04-25T01:11:57.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05f516b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f015dd30 0x7f05f0161a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05f516b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f05f0071d30 0x7f05f006ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05f516b640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05f01628c0 con 0x7f05f00718f0 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05f516b640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f05f0162020 con 0x7f05f0071d30 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05f516b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05f0162300 con 0x7f05f015dd30 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05ef59e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f05f0071d30 0x7f05f006ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05ee59c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f05f00718f0 0x7f05f015ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33178/0 (socket says 172.21.15.33:33178) 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05ee59c640 1 -- 172.21.15.33:0/3394008438 learned_addr learned my addr 172.21.15.33:0/3394008438 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05eed9d640 1 --2- 172.21.15.33:0/3394008438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f015dd30 0x7f05f0161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05ef59e640 1 -- 172.21.15.33:0/3394008438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f015dd30 msgr2=0x7f05f0161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05ef59e640 1 --2- 172.21.15.33:0/3394008438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f015dd30 0x7f05f0161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05ef59e640 1 -- 172.21.15.33:0/3394008438 >> v1:172.21.15.33:6789/0 conn(0x7f05f00718f0 legacy=0x7f05f015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05edd9b640 1 -- 172.21.15.33:0/3394008438 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786078351 0 0) 0x7f05f01628c0 con 0x7f05f00718f0 2024-04-25T01:11:57.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.747+0000 7f05ef59e640 1 -- 172.21.15.33:0/3394008438 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f01625e0 con 0x7f05f0071d30 2024-04-25T01:11:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 --2- 172.21.15.33:0/3394008438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f0071d30 0x7f05f006ccb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097c0 tx=0x7f05e001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec39f08feeefb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05edd9b640 1 -- 172.21.15.33:0/3394008438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00b2070 con 0x7f05f0071d30 2024-04-25T01:11:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05edd9b640 1 -- 172.21.15.33:0/3394008438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e00ad680 con 0x7f05f0071d30 2024-04-25T01:11:57.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05edd9b640 1 -- 172.21.15.33:0/3394008438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00ada50 con 0x7f05f0071d30 2024-04-25T01:11:57.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- 172.21.15.33:0/3394008438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f0071d30 msgr2=0x7f05f006ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 --2- 172.21.15.33:0/3394008438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f0071d30 0x7f05f006ccb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097c0 tx=0x7f05e001c470 comp rx=0 tx=0).stop 2024-04-25T01:11:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- 172.21.15.33:0/3394008438 shutdown_connections 2024-04-25T01:11:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 --2- 172.21.15.33:0/3394008438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f0071d30 0x7f05f006ccb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 --2- 172.21.15.33:0/3394008438 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f015dd30 0x7f05f0161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- 172.21.15.33:0/3394008438 >> 172.21.15.33:0/3394008438 conn(0x7f05f006b3e0 msgr2=0x7f05f006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- 172.21.15.33:0/3394008438 shutdown_connections 2024-04-25T01:11:57.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- 172.21.15.33:0/3394008438 wait complete. 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 Processor -- start 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- start start 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f00718f0 0x7f05f0149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f0071d30 0x7f05f01fae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f015dd30 0x7f05f015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f05f015e320 con 0x7f05f015dd30 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05f016abd0 con 0x7f05f0071d30 2024-04-25T01:11:57.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05f516b640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f05f016ad50 con 0x7f05f00718f0 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f015dd30 0x7f05f015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f015dd30 0x7f05f015d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42892/0 (socket says 172.21.15.33:42892) 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 -- 172.21.15.33:0/2388366538 learned_addr learned my addr 172.21.15.33:0/2388366538 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05eed9d640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f00718f0 0x7f05f0149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ee59c640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f0071d30 0x7f05f01fae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 -- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f00718f0 msgr2=0x7f05f0149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f05f00718f0 0x7f05f0149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 -- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f0071d30 msgr2=0x7f05f01fae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f05f0071d30 0x7f05f01fae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.755+0000 7f05ef59e640 1 -- 172.21.15.33:0/2388366538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f01fa660 con 0x7f05f015dd30 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05ef59e640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f015dd30 0x7f05f015d3a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097c0 tx=0x7f05e0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00b2040 con 0x7f05f015dd30 2024-04-25T01:11:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05e0004250 con 0x7f05f015dd30 2024-04-25T01:11:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f015b840 con 0x7f05f015dd30 2024-04-25T01:11:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00ad970 con 0x7f05f015dd30 2024-04-25T01:11:57.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01fa900 con 0x7f05f015dd30 2024-04-25T01:11:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f05f0069040 con 0x7f05f015dd30 2024-04-25T01:11:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f05e00adb10 con 0x7f05f015dd30 2024-04-25T01:11:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f05dc0678c0 0x7f05dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f05e011fd50 con 0x7f05f015dd30 2024-04-25T01:11:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7f05dc06dd60 0x7f05dc0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 --> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05e0008eb0 con 0x7f05dc06dd60 2024-04-25T01:11:57.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f05e01201e0 con 0x7f05f015dd30 2024-04-25T01:11:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05ee59c640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7f05dc06dd60 0x7f05dc0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05eed9d640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f05dc0678c0 0x7f05dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05ee59c640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7f05dc06dd60 0x7f05dc0701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05eed9d640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f05dc0678c0 0x7f05dc069d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b9d0 tx=0x7f05e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.759+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== osd.2 v2:172.21.15.33:6800/1000748225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f05e0008eb0 con 0x7f05dc06dd60 2024-04-25T01:11:57.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 --> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f05f00aeef0 con 0x7f05dc06dd60 2024-04-25T01:11:57.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05d77fe640 1 -- 172.21.15.33:0/2388366538 <== osd.2 v2:172.21.15.33:6800/1000748225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f05f00aeef0 con 0x7f05dc06dd60 2024-04-25T01:11:57.765 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:11:57.765 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:11:57.765 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:11:57.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7f05dc06dd60 msgr2=0x7f05dc0701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:6800/1000748225,v1:172.21.15.33:6801/1000748225] conn(0x7f05dc06dd60 0x7f05dc0701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f05dc0678c0 msgr2=0x7f05dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f05dc0678c0 0x7f05dc069d80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05e800b9d0 tx=0x7f05e8008040 comp rx=0 tx=0).stop 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f015dd30 msgr2=0x7f05f015d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 --2- 172.21.15.33:0/2388366538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f05f015dd30 0x7f05f015d3a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f05e00097c0 tx=0x7f05e0019210 comp rx=0 tx=0).stop 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05ef59e640 1 -- 172.21.15.33:0/2388366538 reap_dead start 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 shutdown_connections 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 >> 172.21.15.33:0/2388366538 conn(0x7f05f006b3e0 msgr2=0x7f05f015ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 shutdown_connections 2024-04-25T01:11:57.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.775+0000 7f05f516b640 1 -- 172.21.15.33:0/2388366538 wait complete. 2024-04-25T01:11:57.781 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T01:11:57.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9b701640 1 Processor -- start 2024-04-25T01:11:57.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9b701640 1 -- start start 2024-04-25T01:11:57.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9b701640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffb94157be0 0x7ffb9415e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9b701640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 0x7ffb9415a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9b701640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb9415b0e0 con 0x7ffb9406e340 2024-04-25T01:11:57.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9b701640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7ffb9415b3c0 con 0x7ffb94157be0 2024-04-25T01:11:57.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9b701640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9415b6a0 con 0x7ffb940cf2b0 2024-04-25T01:11:57.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb99c9e640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7ffb9406e340 0x7ffb940cd6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33188/0 (socket says 172.21.15.33:33188) 2024-04-25T01:11:57.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb99c9e640 1 -- 172.21.15.33:0/356382219 learned_addr learned my addr 172.21.15.33:0/356382219 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.907+0000 7ffb9949d640 1 --2- 172.21.15.33:0/356382219 >> v2:172.21.15.138:3300/0 conn(0x7ffb94157be0 0x7ffb9415e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 0x7ffb9415a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4273783731 0 0) 0x7ffb9415b0e0 con 0x7ffb9406e340 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb80005740 con 0x7ffb9406e340 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2298694678 0 0) 0x7ffb80005740 con 0x7ffb9406e340 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 msgr2=0x7ffb9415a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 0x7ffb9415a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 >> v2:172.21.15.138:3300/0 conn(0x7ffb94157be0 msgr2=0x7ffb9415e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 --2- 172.21.15.33:0/356382219 >> v2:172.21.15.138:3300/0 conn(0x7ffb94157be0 0x7ffb9415e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9415de00 con 0x7ffb9406e340 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7ffb8c001680 con 0x7ffb9406e340 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 >> v1:172.21.15.33:6789/0 conn(0x7ffb9406e340 legacy=0x7ffb940cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 0x7ffb80005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb80006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80006f60 0x7ffb800073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9415b3c0 con 0x7ffb80006150 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9415b6a0 con 0x7ffb80006f60 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8000dbe0 con 0x7ffb940cf2b0 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb80006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb9949d640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 0x7ffb80005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb99c9e640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80006f60 0x7ffb800073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 -- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 msgr2=0x7ffb80005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940cf2b0 0x7ffb80005bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 -- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80006f60 msgr2=0x7ffb800073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:57.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb80006f60 0x7ffb800073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 -- 172.21.15.33:0/356382219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb80005740 con 0x7ffb80006150 2024-04-25T01:11:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb80006a20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009f90 tx=0x7ffb8801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983366594c3dca09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880b2070 con 0x7ffb80006150 2024-04-25T01:11:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb880a5b40 con 0x7ffb80006150 2024-04-25T01:11:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb87fff640 1 -- 172.21.15.33:0/356382219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88009c10 con 0x7ffb80006150 2024-04-25T01:11:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb9b701640 1 -- 172.21.15.33:0/356382219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 msgr2=0x7ffb80006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb9b701640 1 --2- 172.21.15.33:0/356382219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb80006a20 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009f90 tx=0x7ffb8801cbe0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb99c9e640 1 -- 172.21.15.33:0/356382219 reap_dead start 2024-04-25T01:11:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb9b701640 1 -- 172.21.15.33:0/356382219 shutdown_connections 2024-04-25T01:11:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb9b701640 1 -- 172.21.15.33:0/356382219 >> 172.21.15.33:0/356382219 conn(0x7ffb940b1810 msgr2=0x7ffb940b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb9b701640 1 -- 172.21.15.33:0/356382219 shutdown_connections 2024-04-25T01:11:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.911+0000 7ffb9b701640 1 -- 172.21.15.33:0/356382219 wait complete. 2024-04-25T01:11:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 Processor -- start 2024-04-25T01:11:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 -- start start 2024-04-25T01:11:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb94147a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb940ca5a0 0x7ffb940caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940c5c20 0x7ffb940c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9415eb40 con 0x7ffb80006150 2024-04-25T01:11:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffb94157fc0 con 0x7ffb940ca5a0 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7ffb94147fd0 con 0x7ffb940c5c20 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb94147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb94147a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42914/0 (socket says 172.21.15.33:42914) 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 -- 172.21.15.33:0/1265311762 learned_addr learned my addr 172.21.15.33:0/1265311762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb99c9e640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940c5c20 0x7ffb940c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb940ca5a0 0x7ffb940caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 -- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940c5c20 msgr2=0x7ffb940c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940c5c20 0x7ffb940c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 -- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb940ca5a0 msgr2=0x7ffb940caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb940ca5a0 0x7ffb940caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 -- 172.21.15.33:0/1265311762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb940caf60 con 0x7ffb80006150 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb940ca5a0 0x7ffb940caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9949d640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb94147a90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ffb90011690 tx=0x7ffb90011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb85ffb640 1 -- 172.21.15.33:0/1265311762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9000f9c0 con 0x7ffb80006150 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb85ffb640 1 -- 172.21.15.33:0/1265311762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9000fb60 con 0x7ffb80006150 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb940c6950 con 0x7ffb80006150 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb85ffb640 1 -- 172.21.15.33:0/1265311762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90015460 con 0x7ffb80006150 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb940c6f10 con 0x7ffb80006150 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.915+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb94069040 con 0x7ffb80006150 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.919+0000 7ffb85ffb640 1 -- 172.21.15.33:0/1265311762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb90015600 con 0x7ffb80006150 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.919+0000 7ffb85ffb640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb780678c0 0x7ffb78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:57.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.919+0000 7ffb85ffb640 1 -- 172.21.15.33:0/1265311762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7ffb90086560 con 0x7ffb80006150 2024-04-25T01:11:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.919+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb780678c0 0x7ffb78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:57.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.919+0000 7ffb98c9c640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb780678c0 0x7ffb78069d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb880af000 tx=0x7ffb880a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:57.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:57.923+0000 7ffb85ffb640 1 -- 172.21.15.33:0/1265311762 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7ffb90019030 con 0x7ffb80006150 2024-04-25T01:11:58.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7ffb94072c80 con 0x7ffb780678c0 2024-04-25T01:11:58.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb85ffb640 1 -- 172.21.15.33:0/1265311762 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ffb94072c80 con 0x7ffb780678c0 2024-04-25T01:11:58.069 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T01:11:58.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb780678c0 msgr2=0x7ffb78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb780678c0 0x7ffb78069d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffb880af000 tx=0x7ffb880a7040 comp rx=0 tx=0).stop 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 msgr2=0x7ffb94147a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb94147a90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ffb90011690 tx=0x7ffb90011b50 comp rx=0 tx=0).stop 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 shutdown_connections 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7ffb780678c0 0x7ffb78069d80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7ffb940c5c20 0x7ffb940c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb940ca5a0 0x7ffb940caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 --2- 172.21.15.33:0/1265311762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb80006150 0x7ffb94147a90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 >> 172.21.15.33:0/1265311762 conn(0x7ffb940b1810 msgr2=0x7ffb940b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 shutdown_connections 2024-04-25T01:11:58.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.079+0000 7ffb9b701640 1 -- 172.21.15.33:0/1265311762 wait complete. 2024-04-25T01:11:58.087 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T01:11:58.087 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:11:58.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.223+0000 7f1c212d4640 1 Processor -- start 2024-04-25T01:11:58.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.223+0000 7f1c212d4640 1 -- start start 2024-04-25T01:11:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c1c0b1440 0x7f1c1c161b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 0x7f1c1c06bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1c162a30 con 0x7f1c1c15f000 2024-04-25T01:11:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f1c1c15f4b0 con 0x7f1c1c0b1440 2024-04-25T01:11:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c06b340 con 0x7f1c1c06b7d0 2024-04-25T01:11:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1affd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f1c1c0b1440 0x7f1c1c161b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1a7fc640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f1c1c15f000 0x7f1c1c0b1820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:33190/0 (socket says 172.21.15.33:33190) 2024-04-25T01:11:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1a7fc640 1 -- 172.21.15.33:0/985808859 learned_addr learned my addr 172.21.15.33:0/985808859 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1b7fe640 1 --2- 172.21.15.33:0/985808859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 0x7f1c1c06bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1affd640 1 -- 172.21.15.33:0/985808859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 msgr2=0x7f1c1c06bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1affd640 1 --2- 172.21.15.33:0/985808859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 0x7f1c1c06bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1affd640 1 -- 172.21.15.33:0/985808859 >> v1:172.21.15.33:6789/0 conn(0x7f1c1c15f000 legacy=0x7f1c1c0b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1affd640 1 -- 172.21.15.33:0/985808859 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c162190 con 0x7f1c1c0b1440 2024-04-25T01:11:58.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c19ffb640 1 -- 172.21.15.33:0/985808859 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1747687884 0 0) 0x7f1c1c162a30 con 0x7f1c1c15f000 2024-04-25T01:11:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c1affd640 1 --2- 172.21.15.33:0/985808859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c0b1440 0x7f1c1c161b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0097c0 tx=0x7f1c0c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9172ebeddc5b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c19ffb640 1 -- 172.21.15.33:0/985808859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0a7070 con 0x7f1c1c0b1440 2024-04-25T01:11:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c19ffb640 1 -- 172.21.15.33:0/985808859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0c0b0070 con 0x7f1c1c0b1440 2024-04-25T01:11:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c19ffb640 1 -- 172.21.15.33:0/985808859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c0ab890 con 0x7f1c1c0b1440 2024-04-25T01:11:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- 172.21.15.33:0/985808859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c0b1440 msgr2=0x7f1c1c161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 --2- 172.21.15.33:0/985808859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c0b1440 0x7f1c1c161b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0097c0 tx=0x7f1c0c01c470 comp rx=0 tx=0).stop 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- 172.21.15.33:0/985808859 shutdown_connections 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 --2- 172.21.15.33:0/985808859 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 0x7f1c1c06bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 --2- 172.21.15.33:0/985808859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c0b1440 0x7f1c1c161b00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- 172.21.15.33:0/985808859 >> 172.21.15.33:0/985808859 conn(0x7f1c1c071a80 msgr2=0x7f1c1c0b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- 172.21.15.33:0/985808859 shutdown_connections 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.227+0000 7f1c212d4640 1 -- 172.21.15.33:0/985808859 wait complete. 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 Processor -- start 2024-04-25T01:11:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 -- start start 2024-04-25T01:11:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 0x7f1c1c150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c0b1440 0x7f1c1c0cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c15f000 0x7f1c1c0c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c16b020 con 0x7f1c1c15f000 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1c16aea0 con 0x7f1c1c0b1440 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1c16b1a0 con 0x7f1c1c06b7d0 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c15f000 0x7f1c1c0c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c15f000 0x7f1c1c0c9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42934/0 (socket says 172.21.15.33:42934) 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 -- 172.21.15.33:0/3699475186 learned_addr learned my addr 172.21.15.33:0/3699475186 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1a7fc640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c0b1440 0x7f1c1c0cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1affd640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 0x7f1c1c150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 -- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 msgr2=0x7f1c1c150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f1c1c06b7d0 0x7f1c1c150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 -- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c0b1440 msgr2=0x7f1c1c0cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c1c0b1440 0x7f1c1c0cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 -- 172.21.15.33:0/3699475186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1c156230 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c1b7fe640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c15f000 0x7f1c1c0c9fa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400cc30 tx=0x7f1c1400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400ce40 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c140039a0 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1c156570 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c14003d50 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1c156db0 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.231+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c1c069040 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1c14003f70 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c037fe640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1c080678c0 0x7f1c08069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f1c14086040 con 0x7f1c1c15f000 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c037fe640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f1c0806dd60 0x7f1c080701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 --> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c1400de50 con 0x7f1c0806dd60 2024-04-25T01:11:58.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1c1408d050 con 0x7f1c1c15f000 2024-04-25T01:11:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c1affd640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1c080678c0 0x7f1c08069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c1a7fc640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f1c0806dd60 0x7f1c080701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c1a7fc640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f1c0806dd60 0x7f1c080701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c1affd640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1c080678c0 0x7f1c08069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0af000 tx=0x7f1c0c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.235+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== osd.3 v2:172.21.15.33:6816/2195826364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c1400de50 con 0x7f1c0806dd60 2024-04-25T01:11:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 --> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1c1c0afc20 con 0x7f1c0806dd60 2024-04-25T01:11:58.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c037fe640 1 -- 172.21.15.33:0/3699475186 <== osd.3 v2:172.21.15.33:6816/2195826364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1c1c0afc20 con 0x7f1c0806dd60 2024-04-25T01:11:58.239 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:11:58.239 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:11:58.239 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:11:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f1c0806dd60 msgr2=0x7f1c080701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:6816/2195826364,v1:172.21.15.33:6817/2195826364] conn(0x7f1c0806dd60 0x7f1c080701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1c080678c0 msgr2=0x7f1c08069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f1c080678c0 0x7f1c08069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0af000 tx=0x7f1c0c0a7000 comp rx=0 tx=0).stop 2024-04-25T01:11:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c15f000 msgr2=0x7f1c1c0c9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 --2- 172.21.15.33:0/3699475186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c1c15f000 0x7f1c1c0c9fa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1c1400cc30 tx=0x7f1c1400a5c0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c1b7fe640 1 -- 172.21.15.33:0/3699475186 reap_dead start 2024-04-25T01:11:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 shutdown_connections 2024-04-25T01:11:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 >> 172.21.15.33:0/3699475186 conn(0x7f1c1c071a80 msgr2=0x7f1c1c0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 shutdown_connections 2024-04-25T01:11:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.247+0000 7f1c212d4640 1 -- 172.21.15.33:0/3699475186 wait complete. 2024-04-25T01:11:58.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T01:11:58.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95fb8c640 1 Processor -- start 2024-04-25T01:11:58.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95fb8c640 1 -- start start 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95fb8c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958145470 0x7fb95815c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95fb8c640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9580b5530 0x7fb9580cce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95fb8c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95815f9a0 con 0x7fb9580cf0e0 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95fb8c640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fb95815f750 con 0x7fb9580b5530 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95fb8c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb95815fc80 con 0x7fb958145470 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95e129640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9580b5530 0x7fb9580cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95e129640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fb9580b5530 0x7fb9580cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42936/0 (socket says 172.21.15.33:42936) 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95e129640 1 -- 172.21.15.33:0/150453277 learned_addr learned my addr 172.21.15.33:0/150453277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:58.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95d928640 1 --2- 172.21.15.33:0/150453277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958145470 0x7fb95815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95e129640 1 -- 172.21.15.33:0/150453277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958145470 msgr2=0x7fb95815c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95e129640 1 --2- 172.21.15.33:0/150453277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958145470 0x7fb95815c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95e129640 1 -- 172.21.15.33:0/150453277 >> v1:172.21.15.138:6789/0 conn(0x7fb9580cf0e0 legacy=0x7fb9580b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.383+0000 7fb95e129640 1 -- 172.21.15.33:0/150453277 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95815ff60 con 0x7fb9580b5530 2024-04-25T01:11:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95e129640 1 --2- 172.21.15.33:0/150453277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb9580cce00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb950009e50 tx=0x7fb95001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588187525db1b6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95c926640 1 -- 172.21.15.33:0/150453277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500a5810 con 0x7fb9580b5530 2024-04-25T01:11:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95c926640 1 -- 172.21.15.33:0/150453277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9500a59b0 con 0x7fb9580b5530 2024-04-25T01:11:58.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95c926640 1 -- 172.21.15.33:0/150453277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500a5d80 con 0x7fb9580b5530 2024-04-25T01:11:58.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- 172.21.15.33:0/150453277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 msgr2=0x7fb9580cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/150453277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb9580cce00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb950009e50 tx=0x7fb95001ca30 comp rx=0 tx=0).stop 2024-04-25T01:11:58.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- 172.21.15.33:0/150453277 shutdown_connections 2024-04-25T01:11:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/150453277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb9580cce00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/150453277 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb958145470 0x7fb95815c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- 172.21.15.33:0/150453277 >> 172.21.15.33:0/150453277 conn(0x7fb9580b1810 msgr2=0x7fb9580b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- 172.21.15.33:0/150453277 shutdown_connections 2024-04-25T01:11:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- 172.21.15.33:0/150453277 wait complete. 2024-04-25T01:11:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 Processor -- start 2024-04-25T01:11:58.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- start start 2024-04-25T01:11:58.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb958166290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9580cf0e0 0x7fb95813e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958145470 0x7fb95813a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb958168c70 con 0x7fb9580b5530 2024-04-25T01:11:58.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb958168af0 con 0x7fb958145470 2024-04-25T01:11:58.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95fb8c640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fb958168df0 con 0x7fb9580cf0e0 2024-04-25T01:11:58.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb958166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb958166290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42948/0 (socket says 172.21.15.33:42948) 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 -- 172.21.15.33:0/664112872 learned_addr learned my addr 172.21.15.33:0/664112872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d127640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9580cf0e0 0x7fb95813e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95e129640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958145470 0x7fb95813a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 -- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9580cf0e0 msgr2=0x7fb95813e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9580cf0e0 0x7fb95813e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 -- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958145470 msgr2=0x7fb95813a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958145470 0x7fb95813a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.387+0000 7fb95d928640 1 -- 172.21.15.33:0/664112872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb958142010 con 0x7fb9580b5530 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb95d928640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb958166290 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d800 tx=0x7fb94800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb94e7fc640 1 -- 172.21.15.33:0/664112872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94800b6c0 con 0x7fb9580b5530 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb94e7fc640 1 -- 172.21.15.33:0/664112872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb948019070 con 0x7fb9580b5530 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb94e7fc640 1 -- 172.21.15.33:0/664112872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94800ba60 con 0x7fb9580b5530 2024-04-25T01:11:58.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95813af00 con 0x7fb9580b5530 2024-04-25T01:11:58.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581657b0 con 0x7fb9580b5530 2024-04-25T01:11:58.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb958069040 con 0x7fb9580b5530 2024-04-25T01:11:58.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb94e7fc640 1 -- 172.21.15.33:0/664112872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb94800bc00 con 0x7fb9580b5530 2024-04-25T01:11:58.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb94e7fc640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb93c0678c0 0x7fb93c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb94e7fc640 1 -- 172.21.15.33:0/664112872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fb948085fc0 con 0x7fb9580b5530 2024-04-25T01:11:58.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb95d127640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb93c0678c0 0x7fb93c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.391+0000 7fb95d127640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb93c0678c0 0x7fb93c069d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b17e0 tx=0x7fb954009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.395+0000 7fb94e7fc640 1 -- 172.21.15.33:0/664112872 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fb948056bd0 con 0x7fb9580b5530 2024-04-25T01:11:58.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.559+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb958071400 con 0x7fb93c0678c0 2024-04-25T01:11:58.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb94e7fc640 1 -- 172.21.15.33:0/664112872 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb958071400 con 0x7fb93c0678c0 2024-04-25T01:11:58.552 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T01:11:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb93c0678c0 msgr2=0x7fb93c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb93c0678c0 0x7fb93c069d80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb9580b17e0 tx=0x7fb954009040 comp rx=0 tx=0).stop 2024-04-25T01:11:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 msgr2=0x7fb958166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb958166290 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb94800d800 tx=0x7fb94800dcc0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 shutdown_connections 2024-04-25T01:11:58.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fb93c0678c0 0x7fb93c069d80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb958145470 0x7fb95813a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fb9580cf0e0 0x7fb95813e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 --2- 172.21.15.33:0/664112872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9580b5530 0x7fb958166290 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 >> 172.21.15.33:0/664112872 conn(0x7fb9580b1810 msgr2=0x7fb9580b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 shutdown_connections 2024-04-25T01:11:58.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.563+0000 7fb95fb8c640 1 -- 172.21.15.33:0/664112872 wait complete. 2024-04-25T01:11:58.571 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T01:11:58.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:11:58.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b9ca3640 1 Processor -- start 2024-04-25T01:11:58.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b9ca3640 1 -- start start 2024-04-25T01:11:58.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b9ca3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 0x7f65b40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b9ca3640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f65b40d1080 0x7f65b40b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b9ca3640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65b40d1f20 con 0x7f65b40d0450 2024-04-25T01:11:58.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b9ca3640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f65b40d1cd0 con 0x7f65b40d1080 2024-04-25T01:11:58.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b9ca3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40d26e0 con 0x7f65b40b2e80 2024-04-25T01:11:58.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b37fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f65b40d0450 0x7f65b40d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40466/0 (socket says 172.21.15.33:40466) 2024-04-25T01:11:58.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b37fe640 1 -- 172.21.15.33:0/3285451258 learned_addr learned my addr 172.21.15.33:0/3285451258 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b2ffd640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 0x7f65b40b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3285451258 >> v2:172.21.15.138:3300/0 conn(0x7f65b40d1080 0x7f65b40b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.703+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 213746045 0 0) 0x7f65b40d1f20 con 0x7f65b40d0450 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f659c003530 con 0x7f65b40d0450 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 29748874 0 0) 0x7f659c003530 con 0x7f65b40d0450 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 msgr2=0x7f65b40b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 0x7f65b40b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 >> v2:172.21.15.138:3300/0 conn(0x7f65b40d1080 msgr2=0x7f65b40b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 --2- 172.21.15.33:0/3285451258 >> v2:172.21.15.138:3300/0 conn(0x7f65b40d1080 0x7f65b40b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b40d2200 con 0x7f65b40d0450 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f65a4001680 con 0x7f65b40d0450 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 >> v1:172.21.15.33:6789/0 conn(0x7f65b40d0450 legacy=0x7f65b40d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 0x7f659c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f659c003f90 0x7f659c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f659c004d50 0x7f659c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40d1cd0 con 0x7f659c004d50 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40d26e0 con 0x7f659c003f90 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f659c00ba50 con 0x7f65b40b2e80 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f659c004d50 0x7f659c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b37fe640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 0x7f659c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b2ffd640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f659c003f90 0x7f659c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b3fff640 1 -- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 msgr2=0x7f659c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40b2e80 0x7f659c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b3fff640 1 -- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f659c003f90 msgr2=0x7f659c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f659c003f90 0x7f659c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b3fff640 1 -- 172.21.15.33:0/3285451258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c003530 con 0x7f659c004d50 2024-04-25T01:11:58.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.707+0000 7f65b2ffd640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f659c003f90 0x7f659c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:11:58.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f659c004d50 0x7f659c0051d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009fc0 tx=0x7f65ac01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=370e1fe1a466991a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac0b2070 con 0x7f659c004d50 2024-04-25T01:11:58.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ac0a5cd0 con 0x7f659c004d50 2024-04-25T01:11:58.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b27fc640 1 -- 172.21.15.33:0/3285451258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac009c10 con 0x7f659c004d50 2024-04-25T01:11:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3285451258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f659c004d50 msgr2=0x7f659c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 --2- 172.21.15.33:0/3285451258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f659c004d50 0x7f659c0051d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009fc0 tx=0x7f65ac01ccc0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b3fff640 1 -- 172.21.15.33:0/3285451258 reap_dead start 2024-04-25T01:11:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3285451258 shutdown_connections 2024-04-25T01:11:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3285451258 >> 172.21.15.33:0/3285451258 conn(0x7f65b40d3dd0 msgr2=0x7f65b40d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3285451258 shutdown_connections 2024-04-25T01:11:58.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3285451258 wait complete. 2024-04-25T01:11:58.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 Processor -- start 2024-04-25T01:11:58.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.711+0000 7f65b9ca3640 1 -- start start 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b40cb100 0x7f65b40cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b40b9560 0x7f65b40b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40cb4e0 0x7f65b41f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65b415e8c0 con 0x7f65b40b9560 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f65b40d2c10 con 0x7f65b40cb100 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f65b40d1510 con 0x7f65b40cb4e0 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b40b9560 0x7f65b40b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b40b9560 0x7f65b40b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49642/0 (socket says 172.21.15.33:49642) 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 -- 172.21.15.33:0/3916012250 learned_addr learned my addr 172.21.15.33:0/3916012250 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40cb4e0 0x7f65b41f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b37fe640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b40cb100 0x7f65b40cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 -- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40cb4e0 msgr2=0x7f65b41f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f65b40cb4e0 0x7f65b41f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 -- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b40cb100 msgr2=0x7f65b40cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b40cb100 0x7f65b40cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 -- 172.21.15.33:0/3916012250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b40b84c0 con 0x7f65b40b9560 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b37fe640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65b40cb100 0x7f65b40cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b2ffd640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b40b9560 0x7f65b40b7c70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f65a8011620 tx=0x7f65a8011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a8004220 con 0x7f65b40b9560 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65a80043c0 con 0x7f65b40b9560 2024-04-25T01:11:58.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a800f8c0 con 0x7f65b40b9560 2024-04-25T01:11:58.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b40b87a0 con 0x7f65b40b9560 2024-04-25T01:11:58.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b40b8d60 con 0x7f65b40b9560 2024-04-25T01:11:58.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65b4069040 con 0x7f65b40b9560 2024-04-25T01:11:58.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f65a800fa60 con 0x7f65b40b9560 2024-04-25T01:11:58.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f65a00678f0 0x7f65a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f65a8086190 con 0x7f65b40b9560 2024-04-25T01:11:58.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f65a006dd90 0x7f65a00701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 --> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65a00708a0 con 0x7f65a006dd90 2024-04-25T01:11:58.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f65a808d050 con 0x7f65b40b9560 2024-04-25T01:11:58.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.715+0000 7f65b37fe640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f65a00678f0 0x7f65a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.719+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f65a006dd90 0x7f65a00701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.719+0000 7f65b37fe640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f65a00678f0 0x7f65a0069db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f65a4000c10 tx=0x7f65a4000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.719+0000 7f65b3fff640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f65a006dd90 0x7f65a00701d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.719+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== osd.4 v2:172.21.15.138:6816/100119801 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65a00708a0 con 0x7f65a006dd90 2024-04-25T01:11:58.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.731+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 --> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f65b40d0ad0 con 0x7f65a006dd90 2024-04-25T01:11:58.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.731+0000 7f659bfff640 1 -- 172.21.15.33:0/3916012250 <== osd.4 v2:172.21.15.138:6816/100119801 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f65b40d0ad0 con 0x7f65a006dd90 2024-04-25T01:11:58.723 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:11:58.724 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:11:58.724 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:11:58.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.731+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f65a006dd90 msgr2=0x7f65a00701d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.731+0000 7f65b9ca3640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6816/100119801,v1:172.21.15.138:6817/100119801] conn(0x7f65a006dd90 0x7f65a00701d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f65a00678f0 msgr2=0x7f65a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f65a00678f0 0x7f65a0069db0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f65a4000c10 tx=0x7f65a4000ee0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b40b9560 msgr2=0x7f65b40b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 --2- 172.21.15.33:0/3916012250 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65b40b9560 0x7f65b40b7c70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f65a8011620 tx=0x7f65a8011ae0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b3fff640 1 -- 172.21.15.33:0/3916012250 reap_dead start 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 shutdown_connections 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 >> 172.21.15.33:0/3916012250 conn(0x7f65b40d3dd0 msgr2=0x7f65b40d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 shutdown_connections 2024-04-25T01:11:58.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.735+0000 7f65b9ca3640 1 -- 172.21.15.33:0/3916012250 wait complete. 2024-04-25T01:11:58.738 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T01:11:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.855+0000 7f8dbca98640 1 Processor -- start 2024-04-25T01:11:58.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.855+0000 7f8dbca98640 1 -- start start 2024-04-25T01:11:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.855+0000 7f8dbca98640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8db815aaf0 0x7f8db80af940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.855+0000 7f8dbca98640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8db80b3bc0 0x7f8db8071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.855+0000 7f8dbca98640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8db80b4110 con 0x7f8db8072370 2024-04-25T01:11:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.855+0000 7f8dbca98640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f8db80b43f0 con 0x7f8db80b3bc0 2024-04-25T01:11:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.855+0000 7f8dbca98640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f8db80b46d0 con 0x7f8db815aaf0 2024-04-25T01:11:58.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8db80b3bc0 0x7f8db8071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f8db80b3bc0 0x7f8db8071e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49658/0 (socket says 172.21.15.33:49658) 2024-04-25T01:11:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 -- 172.21.15.33:0/2523851611 learned_addr learned my addr 172.21.15.33:0/2523851611 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 -- 172.21.15.33:0/2523851611 >> v2:172.21.15.138:3300/0 conn(0x7f8db815aaf0 msgr2=0x7f8db80af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db659c640 1 --2- 172.21.15.33:0/2523851611 >> v2:172.21.15.138:3300/0 conn(0x7f8db815aaf0 0x7f8db80af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 --2- 172.21.15.33:0/2523851611 >> v2:172.21.15.138:3300/0 conn(0x7f8db815aaf0 0x7f8db80af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 -- 172.21.15.33:0/2523851611 >> v1:172.21.15.138:6789/0 conn(0x7f8db8072370 legacy=0x7f8db80add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:58.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 -- 172.21.15.33:0/2523851611 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db80b49b0 con 0x7f8db80b3bc0 2024-04-25T01:11:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db5d9b640 1 --2- 172.21.15.33:0/2523851611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db80b3bc0 0x7f8db8071e30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8db80b3450 tx=0x7f8da801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81411e7a2f6d8d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db559a640 1 -- 172.21.15.33:0/2523851611 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da80b0070 con 0x7f8db80b3bc0 2024-04-25T01:11:58.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db559a640 1 -- 172.21.15.33:0/2523851611 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8da80ab600 con 0x7f8db80b3bc0 2024-04-25T01:11:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8db559a640 1 -- 172.21.15.33:0/2523851611 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da80ab9d0 con 0x7f8db80b3bc0 2024-04-25T01:11:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 -- 172.21.15.33:0/2523851611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db80b3bc0 msgr2=0x7f8db8071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 --2- 172.21.15.33:0/2523851611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db80b3bc0 0x7f8db8071e30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8db80b3450 tx=0x7f8da801c4e0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 -- 172.21.15.33:0/2523851611 shutdown_connections 2024-04-25T01:11:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 --2- 172.21.15.33:0/2523851611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db80b3bc0 0x7f8db8071e30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 --2- 172.21.15.33:0/2523851611 >> v2:172.21.15.138:3300/0 conn(0x7f8db815aaf0 0x7f8db80af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 -- 172.21.15.33:0/2523851611 >> 172.21.15.33:0/2523851611 conn(0x7f8db80b28c0 msgr2=0x7f8db80b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 -- 172.21.15.33:0/2523851611 shutdown_connections 2024-04-25T01:11:58.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.859+0000 7f8dbca98640 1 -- 172.21.15.33:0/2523851611 wait complete. 2024-04-25T01:11:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 Processor -- start 2024-04-25T01:11:58.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 -- start start 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db8072370 0x7f8db80b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db80b3bc0 0x7f8db8164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db815aaf0 0x7f8db81470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8db815aed0 con 0x7f8db8072370 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8db806f640 con 0x7f8db815aaf0 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f8db8152bd0 con 0x7f8db80b3bc0 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db8072370 0x7f8db80b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db8072370 0x7f8db80b1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49668/0 (socket says 172.21.15.33:49668) 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 -- 172.21.15.33:0/4234797891 learned_addr learned my addr 172.21.15.33:0/4234797891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db5d9b640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db80b3bc0 0x7f8db8164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db6d9d640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db815aaf0 0x7f8db81470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 -- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db80b3bc0 msgr2=0x7f8db8164ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db80b3bc0 0x7f8db8164ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 -- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db815aaf0 msgr2=0x7f8db81470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db815aaf0 0x7f8db81470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 -- 172.21.15.33:0/4234797891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db8152e20 con 0x7f8db8072370 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8db659c640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db8072370 0x7f8db80b1f60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8db000ce10 tx=0x7f8db000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8da6ffd640 1 -- 172.21.15.33:0/4234797891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0003ba0 con 0x7f8db8072370 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8da6ffd640 1 -- 172.21.15.33:0/4234797891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8db0003d40 con 0x7f8db8072370 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db8153100 con 0x7f8db8072370 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8da6ffd640 1 -- 172.21.15.33:0/4234797891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db0004240 con 0x7f8db8072370 2024-04-25T01:11:58.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db81646e0 con 0x7f8db8072370 2024-04-25T01:11:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db8069040 con 0x7f8db8072370 2024-04-25T01:11:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.863+0000 7f8da6ffd640 1 -- 172.21.15.33:0/4234797891 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8db00043e0 con 0x7f8db8072370 2024-04-25T01:11:58.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.867+0000 7f8da6ffd640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d980678c0 0x7f8d98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.867+0000 7f8da6ffd640 1 -- 172.21.15.33:0/4234797891 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f8db0086650 con 0x7f8db8072370 2024-04-25T01:11:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.867+0000 7f8db5d9b640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d980678c0 0x7f8d98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:58.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.867+0000 7f8db5d9b640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d980678c0 0x7f8d98069d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8da80ad000 tx=0x7f8da8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:58.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:58.871+0000 7f8da6ffd640 1 -- 172.21.15.33:0/4234797891 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f8db0057310 con 0x7f8db8072370 2024-04-25T01:11:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.027+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f8db80d1620 con 0x7f8d980678c0 2024-04-25T01:11:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.027+0000 7f8da6ffd640 1 -- 172.21.15.33:0/4234797891 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8db80d1620 con 0x7f8d980678c0 2024-04-25T01:11:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d980678c0 msgr2=0x7f8d98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d980678c0 0x7f8d98069d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8da80ad000 tx=0x7f8da8008040 comp rx=0 tx=0).stop 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db8072370 msgr2=0x7f8db80b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db8072370 0x7f8db80b1f60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8db000ce10 tx=0x7f8db000a660 comp rx=0 tx=0).stop 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 shutdown_connections 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f8d980678c0 0x7f8d98069d80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8db815aaf0 0x7f8db81470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f8db80b3bc0 0x7f8db8164ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 --2- 172.21.15.33:0/4234797891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8db8072370 0x7f8db80b1f60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 >> 172.21.15.33:0/4234797891 conn(0x7f8db80b28c0 msgr2=0x7f8db80b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 shutdown_connections 2024-04-25T01:11:59.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.031+0000 7f8dbca98640 1 -- 172.21.15.33:0/4234797891 wait complete. 2024-04-25T01:11:59.037 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T01:11:59.037 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:11:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.163+0000 7f6a8f677640 1 Processor -- start 2024-04-25T01:11:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.163+0000 7f6a8f677640 1 -- start start 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6a881625a0 0x7f6a880d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a880b0fe0 con 0x7f6a8815e0c0 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a880d0e20 con 0x7f6a8815e4a0 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6a880d1100 con 0x7f6a881625a0 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8d413640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6a8815e0c0 0x7f6a88161e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40470/0 (socket says 172.21.15.33:40470) 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8d413640 1 -- 172.21.15.33:0/233883911 learned_addr learned my addr 172.21.15.33:0/233883911 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8cc12640 1 --2- 172.21.15.33:0/233883911 >> v2:172.21.15.33:3300/0 conn(0x7f6a881625a0 0x7f6a880d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a7ffff640 1 -- 172.21.15.33:0/233883911 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688141117 0 0) 0x7f6a84001120 con 0x7f6a8815e0c0 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a7ffff640 1 -- 172.21.15.33:0/233883911 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a74003530 con 0x7f6a8815e0c0 2024-04-25T01:11:59.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8cc12640 1 -- 172.21.15.33:0/233883911 >> v1:172.21.15.138:6789/0 conn(0x7f6a8815e4a0 legacy=0x7f6a880af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8cc12640 1 -- 172.21.15.33:0/233883911 >> v1:172.21.15.33:6789/0 conn(0x7f6a8815e0c0 legacy=0x7f6a88161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8cc12640 1 -- 172.21.15.33:0/233883911 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a880aec60 con 0x7f6a881625a0 2024-04-25T01:11:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8cc12640 1 --2- 172.21.15.33:0/233883911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a881625a0 0x7f6a880d0840 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6a780097c0 tx=0x7f6a7801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae41253334e099ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a7ffff640 1 -- 172.21.15.33:0/233883911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780b0070 con 0x7f6a881625a0 2024-04-25T01:11:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a7ffff640 1 -- 172.21.15.33:0/233883911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a780ab600 con 0x7f6a881625a0 2024-04-25T01:11:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a7ffff640 1 -- 172.21.15.33:0/233883911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a780ab9d0 con 0x7f6a881625a0 2024-04-25T01:11:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- 172.21.15.33:0/233883911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a881625a0 msgr2=0x7f6a880d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 --2- 172.21.15.33:0/233883911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a881625a0 0x7f6a880d0840 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f6a780097c0 tx=0x7f6a7801c4a0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- 172.21.15.33:0/233883911 shutdown_connections 2024-04-25T01:11:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 --2- 172.21.15.33:0/233883911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a881625a0 0x7f6a880d0840 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- 172.21.15.33:0/233883911 >> 172.21.15.33:0/233883911 conn(0x7f6a880d1ca0 msgr2=0x7f6a880d20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- 172.21.15.33:0/233883911 shutdown_connections 2024-04-25T01:11:59.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.167+0000 7f6a8f677640 1 -- 172.21.15.33:0/233883911 wait complete. 2024-04-25T01:11:59.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 Processor -- start 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 -- start start 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8815e0c0 0x7f6a8814d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8815e4a0 0x7f6a880b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a881625a0 0x7f6a880b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880b0790 con 0x7f6a8815e0c0 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880b0610 con 0x7f6a8815e4a0 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6a8816c1e0 con 0x7f6a881625a0 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8815e0c0 0x7f6a8814d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8815e0c0 0x7f6a8814d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49686/0 (socket says 172.21.15.33:49686) 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 -- 172.21.15.33:0/2468237963 learned_addr learned my addr 172.21.15.33:0/2468237963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 -- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a881625a0 msgr2=0x7f6a880b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6a881625a0 0x7f6a880b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 -- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8815e4a0 msgr2=0x7f6a880b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8815e4a0 0x7f6a880b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 -- 172.21.15.33:0/2468237963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a88162d60 con 0x7f6a8815e0c0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8d413640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8815e0c0 0x7f6a8814d150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400c730 tx=0x7f6a8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84017070 con 0x7f6a8815e0c0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a84005ac0 con 0x7f6a8815e0c0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8814d690 con 0x7f6a8815e0c0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84007070 con 0x7f6a8815e0c0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a88152790 con 0x7f6a8815e0c0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a88153130 con 0x7f6a8815e0c0 2024-04-25T01:11:59.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.171+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a8401a070 con 0x7f6a8815e0c0 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a7dffb640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a6c0678f0 0x7f6a6c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f6a84085320 con 0x7f6a8815e0c0 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a7dffb640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f6a6c06dd90 0x7f6a6c0701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 --> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a8400ae50 con 0x7f6a6c06dd90 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6a840857b0 con 0x7f6a8815e0c0 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a8cc12640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a6c0678f0 0x7f6a6c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a8dc14640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f6a6c06dd90 0x7f6a6c0701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a8dc14640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f6a6c06dd90 0x7f6a6c0701d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a8cc12640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a6c0678f0 0x7f6a6c069db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a780ad000 tx=0x7f6a78004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.174+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== osd.5 v2:172.21.15.138:6801/1955200876 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6a8400ae50 con 0x7f6a6c06dd90 2024-04-25T01:11:59.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.186+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 --> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6a88152c20 con 0x7f6a6c06dd90 2024-04-25T01:11:59.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a7dffb640 1 -- 172.21.15.33:0/2468237963 <== osd.5 v2:172.21.15.138:6801/1955200876 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6a88152c20 con 0x7f6a6c06dd90 2024-04-25T01:11:59.180 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:11:59.181 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:11:59.181 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f6a6c06dd90 msgr2=0x7f6a6c0701d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6801/1955200876,v1:172.21.15.138:6804/1955200876] conn(0x7f6a6c06dd90 0x7f6a6c0701d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a6c0678f0 msgr2=0x7f6a6c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6a6c0678f0 0x7f6a6c069db0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6a780ad000 tx=0x7f6a78004740 comp rx=0 tx=0).stop 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8815e0c0 msgr2=0x7f6a8814d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 --2- 172.21.15.33:0/2468237963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8815e0c0 0x7f6a8814d150 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400c730 tx=0x7f6a8400cbf0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8dc14640 1 -- 172.21.15.33:0/2468237963 reap_dead start 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 shutdown_connections 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 >> 172.21.15.33:0/2468237963 conn(0x7f6a880d1ca0 msgr2=0x7f6a8815cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 shutdown_connections 2024-04-25T01:11:59.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.190+0000 7f6a8f677640 1 -- 172.21.15.33:0/2468237963 wait complete. 2024-04-25T01:11:59.195 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T01:11:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98ab872640 1 Processor -- start 2024-04-25T01:11:59.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98ab872640 1 -- start start 2024-04-25T01:11:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98ab872640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98a4140a70 0x7f98a4143510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98ab872640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f98a40cf230 0x7f98a4143a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98ab872640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98a40b48f0 con 0x7f98a40b5950 2024-04-25T01:11:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98ab872640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40b54c0 con 0x7f98a4140a70 2024-04-25T01:11:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98ab872640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f98a40b4b90 con 0x7f98a40cf230 2024-04-25T01:11:59.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98a960e640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f98a4140a70 0x7f98a4143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98a8e0d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f98a40b5950 0x7f98a40ceb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40484/0 (socket says 172.21.15.33:40484) 2024-04-25T01:11:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.318+0000 7f98a8e0d640 1 -- 172.21.15.33:0/187027075 learned_addr learned my addr 172.21.15.33:0/187027075 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98a960e640 1 -- 172.21.15.33:0/187027075 >> v2:172.21.15.138:3300/0 conn(0x7f98a40cf230 msgr2=0x7f98a4143a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98a960e640 1 --2- 172.21.15.33:0/187027075 >> v2:172.21.15.138:3300/0 conn(0x7f98a40cf230 0x7f98a4143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98a960e640 1 -- 172.21.15.33:0/187027075 >> v1:172.21.15.33:6789/0 conn(0x7f98a40b5950 legacy=0x7f98a40ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f989bfff640 1 -- 172.21.15.33:0/187027075 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2943912584 0 0) 0x7f98a40b48f0 con 0x7f98a40b5950 2024-04-25T01:11:59.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98a960e640 1 -- 172.21.15.33:0/187027075 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a40b4e70 con 0x7f98a4140a70 2024-04-25T01:11:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98a960e640 1 --2- 172.21.15.33:0/187027075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a4140a70 0x7f98a4143510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f98a40b5640 tx=0x7f98a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf89d103840df3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f989bfff640 1 -- 172.21.15.33:0/187027075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00b1070 con 0x7f98a4140a70 2024-04-25T01:11:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f989bfff640 1 -- 172.21.15.33:0/187027075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a00ac5f0 con 0x7f98a4140a70 2024-04-25T01:11:59.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f989bfff640 1 -- 172.21.15.33:0/187027075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a00ac9c0 con 0x7f98a4140a70 2024-04-25T01:11:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 -- 172.21.15.33:0/187027075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a4140a70 msgr2=0x7f98a4143510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 --2- 172.21.15.33:0/187027075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a4140a70 0x7f98a4143510 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f98a40b5640 tx=0x7f98a001c480 comp rx=0 tx=0).stop 2024-04-25T01:11:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 -- 172.21.15.33:0/187027075 shutdown_connections 2024-04-25T01:11:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 --2- 172.21.15.33:0/187027075 >> v2:172.21.15.138:3300/0 conn(0x7f98a40cf230 0x7f98a4143a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 --2- 172.21.15.33:0/187027075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a4140a70 0x7f98a4143510 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 -- 172.21.15.33:0/187027075 >> 172.21.15.33:0/187027075 conn(0x7f98a40af780 msgr2=0x7f98a40afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 -- 172.21.15.33:0/187027075 shutdown_connections 2024-04-25T01:11:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.322+0000 7f98ab872640 1 -- 172.21.15.33:0/187027075 wait complete. 2024-04-25T01:11:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 Processor -- start 2024-04-25T01:11:59.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 -- start start 2024-04-25T01:11:59.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a40b5950 0x7f98a4154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40cf230 0x7f98a4137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a4140a70 0x7f98a415a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98a40ad840 con 0x7f98a40cf230 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98a415f010 con 0x7f98a4140a70 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f98a40ad9c0 con 0x7f98a40b5950 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40cf230 0x7f98a4137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40cf230 0x7f98a4137fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49708/0 (socket says 172.21.15.33:49708) 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 -- 172.21.15.33:0/3240476207 learned_addr learned my addr 172.21.15.33:0/3240476207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a9e0f640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a4140a70 0x7f98a415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a960e640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a40b5950 0x7f98a4154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 -- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a40b5950 msgr2=0x7f98a4154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a40b5950 0x7f98a4154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 -- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a4140a70 msgr2=0x7f98a415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a4140a70 0x7f98a415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 -- 172.21.15.33:0/3240476207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a4158e80 con 0x7f98a40cf230 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a9e0f640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a4140a70 0x7f98a415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a960e640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a40b5950 0x7f98a4154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98a8e0d640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40cf230 0x7f98a4137fd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f989400c550 tx=0x7f989400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f9899ffb640 1 -- 172.21.15.33:0/3240476207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894017070 con 0x7f98a40cf230 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f9899ffb640 1 -- 172.21.15.33:0/3240476207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9894005d10 con 0x7f98a40cf230 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f9899ffb640 1 -- 172.21.15.33:0/3240476207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9894007330 con 0x7f98a40cf230 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a4159160 con 0x7f98a40cf230 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a4159720 con 0x7f98a40cf230 2024-04-25T01:11:59.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4069040 con 0x7f98a40cf230 2024-04-25T01:11:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f9899ffb640 1 -- 172.21.15.33:0/3240476207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f989401a070 con 0x7f98a40cf230 2024-04-25T01:11:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.326+0000 7f9899ffb640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f98880678c0 0x7f9888069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.330+0000 7f9899ffb640 1 -- 172.21.15.33:0/3240476207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f9894085030 con 0x7f98a40cf230 2024-04-25T01:11:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.330+0000 7f98a960e640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f98880678c0 0x7f9888069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.330+0000 7f98a960e640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f98880678c0 0x7f9888069d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f98a00ae000 tx=0x7f98a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.334+0000 7f9899ffb640 1 -- 172.21.15.33:0/3240476207 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f9894055ce0 con 0x7f98a40cf230 2024-04-25T01:11:59.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.486+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f98a415caa0 con 0x7f98880678c0 2024-04-25T01:11:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f9899ffb640 1 -- 172.21.15.33:0/3240476207 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f98a415caa0 con 0x7f98880678c0 2024-04-25T01:11:59.480 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T01:11:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f98880678c0 msgr2=0x7f9888069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f98880678c0 0x7f9888069d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f98a00ae000 tx=0x7f98a0008040 comp rx=0 tx=0).stop 2024-04-25T01:11:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40cf230 msgr2=0x7f98a4137fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40cf230 0x7f98a4137fd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f989400c550 tx=0x7f989400ca10 comp rx=0 tx=0).stop 2024-04-25T01:11:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 shutdown_connections 2024-04-25T01:11:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f98880678c0 0x7f9888069d80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a4140a70 0x7f98a415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a40cf230 0x7f98a4137fd0 secure :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f989400c550 tx=0x7f989400ca10 comp rx=0 tx=0).stop 2024-04-25T01:11:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 --2- 172.21.15.33:0/3240476207 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f98a40b5950 0x7f98a4154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 >> 172.21.15.33:0/3240476207 conn(0x7f98a40af780 msgr2=0x7f98a40b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 shutdown_connections 2024-04-25T01:11:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.490+0000 7f98ab872640 1 -- 172.21.15.33:0/3240476207 wait complete. 2024-04-25T01:11:59.498 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T01:11:59.498 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:11:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.630+0000 7f3f1c236640 1 Processor -- start 2024-04-25T01:11:59.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.630+0000 7f3f1c236640 1 -- start start 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f1c236640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f141553f0 0x7f3f14066f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f1c236640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3f1415c070 0x7f3f14072910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f1c236640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3f1406c750 0x7f3f14072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f1c236640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1415b2d0 con 0x7f3f1415c070 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f1c236640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f3f1406ccc0 con 0x7f3f1406c750 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f1c236640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1415b590 con 0x7f3f141553f0 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3f1415c070 0x7f3f14072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f3f1415c070 0x7f3f14072910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49710/0 (socket says 172.21.15.33:49710) 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 -- 172.21.15.33:0/1700539988 learned_addr learned my addr 172.21.15.33:0/1700539988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f1a7d3640 1 --2- 172.21.15.33:0/1700539988 >> v2:172.21.15.138:3300/0 conn(0x7f3f1406c750 0x7f3f14072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f19fd2640 1 --2- 172.21.15.33:0/1700539988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f141553f0 0x7f3f14066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 -- 172.21.15.33:0/1700539988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f141553f0 msgr2=0x7f3f14066f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 --2- 172.21.15.33:0/1700539988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f141553f0 0x7f3f14066f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 -- 172.21.15.33:0/1700539988 >> v2:172.21.15.138:3300/0 conn(0x7f3f1406c750 msgr2=0x7f3f14072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 --2- 172.21.15.33:0/1700539988 >> v2:172.21.15.138:3300/0 conn(0x7f3f1406c750 0x7f3f14072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 -- 172.21.15.33:0/1700539988 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f1415b870 con 0x7f3f1415c070 2024-04-25T01:11:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f197d1640 1 --2- 172.21.15.33:0/1700539988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1415c070 0x7f3f14072910 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3f140b5880 tx=0x7f3f0401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edaed5e92c03d64a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f18fd0640 1 -- 172.21.15.33:0/1700539988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f040b0070 con 0x7f3f1415c070 2024-04-25T01:11:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f18fd0640 1 -- 172.21.15.33:0/1700539988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f040ab600 con 0x7f3f1415c070 2024-04-25T01:11:59.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.634+0000 7f3f18fd0640 1 -- 172.21.15.33:0/1700539988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f040ab9d0 con 0x7f3f1415c070 2024-04-25T01:11:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/1700539988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1415c070 msgr2=0x7f3f14072910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 --2- 172.21.15.33:0/1700539988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1415c070 0x7f3f14072910 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f3f140b5880 tx=0x7f3f0401c520 comp rx=0 tx=0).stop 2024-04-25T01:11:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/1700539988 shutdown_connections 2024-04-25T01:11:59.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 --2- 172.21.15.33:0/1700539988 >> v2:172.21.15.138:3300/0 conn(0x7f3f1406c750 0x7f3f14072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 --2- 172.21.15.33:0/1700539988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f1415c070 0x7f3f14072910 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 --2- 172.21.15.33:0/1700539988 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f141553f0 0x7f3f14066f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/1700539988 >> 172.21.15.33:0/1700539988 conn(0x7f3f140b3df0 msgr2=0x7f3f140b5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/1700539988 shutdown_connections 2024-04-25T01:11:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/1700539988 wait complete. 2024-04-25T01:11:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 Processor -- start 2024-04-25T01:11:59.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- start start 2024-04-25T01:11:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1406c750 0x7f3f14144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f141553f0 0x7f3f14159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f1415c070 0x7f3f14146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1406bdb0 con 0x7f3f141553f0 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3f1406bc30 con 0x7f3f1406c750 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f3f1406bf30 con 0x7f3f1415c070 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f141553f0 0x7f3f14159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f141553f0 0x7f3f14159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49718/0 (socket says 172.21.15.33:49718) 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 -- 172.21.15.33:0/2109311525 learned_addr learned my addr 172.21.15.33:0/2109311525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f19fd2640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1406c750 0x7f3f14144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1a7d3640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f1415c070 0x7f3f14146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 -- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f1415c070 msgr2=0x7f3f14146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f3f1415c070 0x7f3f14146f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 -- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1406c750 msgr2=0x7f3f14144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3f1406c750 0x7f3f14144470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 -- 172.21.15.33:0/2109311525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f14141d30 con 0x7f3f141553f0 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f197d1640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f141553f0 0x7f3f14159f50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f040ae040 tx=0x7f3f040192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f040b0050 con 0x7f3f141553f0 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f040bc070 con 0x7f3f141553f0 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f14142010 con 0x7f3f141553f0 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f040b5a50 con 0x7f3f141553f0 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f1414bab0 con 0x7f3f141553f0 2024-04-25T01:11:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.638+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f14069040 con 0x7f3f141553f0 2024-04-25T01:11:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3f040ab4c0 con 0x7f3f141553f0 2024-04-25T01:11:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f0a7fc640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3ef80678c0 0x7f3ef8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f3f04120e70 con 0x7f3f141553f0 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f0a7fc640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7f3ef806dd60 0x7f3ef80701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 --> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3ef8070870 con 0x7f3ef806dd60 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f3f04121220 con 0x7f3f141553f0 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f19fd2640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3ef80678c0 0x7f3ef8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f1a7d3640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7f3ef806dd60 0x7f3ef80701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f1a7d3640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7f3ef806dd60 0x7f3ef80701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f19fd2640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3ef80678c0 0x7f3ef8069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000b930 tx=0x7f3f10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.642+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== osd.6 v2:172.21.15.138:6824/1315598862 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3ef8070870 con 0x7f3ef806dd60 2024-04-25T01:11:59.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 --> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f14071c10 con 0x7f3ef806dd60 2024-04-25T01:11:59.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f0a7fc640 1 -- 172.21.15.33:0/2109311525 <== osd.6 v2:172.21.15.138:6824/1315598862 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3f14071c10 con 0x7f3ef806dd60 2024-04-25T01:11:59.650 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:11:59.650 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:11:59.650 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7f3ef806dd60 msgr2=0x7f3ef80701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6824/1315598862,v1:172.21.15.138:6825/1315598862] conn(0x7f3ef806dd60 0x7f3ef80701a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3ef80678c0 msgr2=0x7f3ef8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f3ef80678c0 0x7f3ef8069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000b930 tx=0x7f3f10008040 comp rx=0 tx=0).stop 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f141553f0 msgr2=0x7f3f14159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 --2- 172.21.15.33:0/2109311525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3f141553f0 0x7f3f14159f50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f040ae040 tx=0x7f3f040192c0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1a7d3640 1 -- 172.21.15.33:0/2109311525 reap_dead start 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 shutdown_connections 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 >> 172.21.15.33:0/2109311525 conn(0x7f3f140b3df0 msgr2=0x7f3f140d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 shutdown_connections 2024-04-25T01:11:59.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.658+0000 7f3f1c236640 1 -- 172.21.15.33:0/2109311525 wait complete. 2024-04-25T01:11:59.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T01:11:59.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecd5554640 1 Processor -- start 2024-04-25T01:11:59.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecd5554640 1 -- start start 2024-04-25T01:11:59.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecd5554640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fecd00d4950 0x7fecd00d0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecd5554640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 0x7fecd00cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecd5554640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecd00d25b0 con 0x7fecd015dfa0 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecd5554640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fecd00d2890 con 0x7fecd00d4950 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecd5554640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fecd00d2b70 con 0x7fecd00cee60 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecceffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fecd015dfa0 0x7fecd00d0720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40498/0 (socket says 172.21.15.33:40498) 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecceffd640 1 -- 172.21.15.33:0/2422067717 learned_addr learned my addr 172.21.15.33:0/2422067717 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecce7fc640 1 --2- 172.21.15.33:0/2422067717 >> v2:172.21.15.138:3300/0 conn(0x7fecd00d4950 0x7fecd00d0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccf7fe640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 0x7fecd00cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2476682832 0 0) 0x7fecd00d25b0 con 0x7fecd015dfa0 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fecb8003530 con 0x7fecd015dfa0 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2962442150 0 0) 0x7fecb8003530 con 0x7fecd015dfa0 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 msgr2=0x7fecd00cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 0x7fecd00cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 >> v2:172.21.15.138:3300/0 conn(0x7fecd00d4950 msgr2=0x7fecd00d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 --2- 172.21.15.33:0/2422067717 >> v2:172.21.15.138:3300/0 conn(0x7fecd00d4950 0x7fecd00d0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd00b3330 con 0x7fecd015dfa0 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7fecc4001680 con 0x7fecd015dfa0 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 >> v1:172.21.15.33:6789/0 conn(0x7fecd015dfa0 legacy=0x7fecd00d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb80039c0 0x7fecb8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 0x7fecb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecb8004d50 0x7fecb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecd00d2890 con 0x7fecb8004d50 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fecd00d2b70 con 0x7fecb80039c0 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fecb800b9d0 con 0x7fecd00cee60 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccf7fe640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecb8004d50 0x7fecb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecceffd640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb80039c0 0x7fecb8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7fecce7fc640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 0x7fecb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccf7fe640 1 -- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 msgr2=0x7fecb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccf7fe640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecd00cee60 0x7fecb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccf7fe640 1 -- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb80039c0 msgr2=0x7fecb8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccf7fe640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecb80039c0 0x7fecb8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.782+0000 7feccf7fe640 1 -- 172.21.15.33:0/2422067717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb8003530 con 0x7fecb8004d50 2024-04-25T01:11:59.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7feccf7fe640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecb8004d50 0x7fecb80051d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009fa0 tx=0x7fecc801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5460dad5f3fc211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc80b2070 con 0x7fecb8004d50 2024-04-25T01:11:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc80a5d10 con 0x7fecb8004d50 2024-04-25T01:11:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7feccdffb640 1 -- 172.21.15.33:0/2422067717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc8009bf0 con 0x7fecb8004d50 2024-04-25T01:11:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 -- 172.21.15.33:0/2422067717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecb8004d50 msgr2=0x7fecb80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 --2- 172.21.15.33:0/2422067717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecb8004d50 0x7fecb80051d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009fa0 tx=0x7fecc801cc40 comp rx=0 tx=0).stop 2024-04-25T01:11:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7feccf7fe640 1 -- 172.21.15.33:0/2422067717 reap_dead start 2024-04-25T01:11:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 -- 172.21.15.33:0/2422067717 shutdown_connections 2024-04-25T01:11:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 -- 172.21.15.33:0/2422067717 >> 172.21.15.33:0/2422067717 conn(0x7fecd00d3e30 msgr2=0x7fecd00d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 -- 172.21.15.33:0/2422067717 shutdown_connections 2024-04-25T01:11:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 -- 172.21.15.33:0/2422067717 wait complete. 2024-04-25T01:11:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 Processor -- start 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.786+0000 7fecd5554640 1 -- start start 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb8004d50 0x7fecd00cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecd00cdbf0 0x7fecd01f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecd006cc60 0x7fecd006d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecd015e500 con 0x7fecd00cdbf0 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fecd015e380 con 0x7fecd006cc60 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fecd00b0a30 con 0x7fecb8004d50 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecd00cdbf0 0x7fecd01f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecd00cdbf0 0x7fecd01f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49736/0 (socket says 172.21.15.33:49736) 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 -- 172.21.15.33:0/270411871 learned_addr learned my addr 172.21.15.33:0/270411871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:11:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecceffd640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb8004d50 0x7fecd00cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7feccf7fe640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecd006cc60 0x7fecd006d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 -- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb8004d50 msgr2=0x7fecd00cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb8004d50 0x7fecd00cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 -- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecd006cc60 msgr2=0x7fecd006d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecd006cc60 0x7fecd006d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 -- 172.21.15.33:0/270411871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecd009e0c0 con 0x7fecd00cdbf0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecce7fc640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecd00cdbf0 0x7fecd01f4610 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fecc0011630 tx=0x7fecc0011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecb77fe640 1 -- 172.21.15.33:0/270411871 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc0004220 con 0x7fecd00cdbf0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecb77fe640 1 -- 172.21.15.33:0/270411871 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecc00043c0 con 0x7fecd00cdbf0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecd01f4b50 con 0x7fecd00cdbf0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecb77fe640 1 -- 172.21.15.33:0/270411871 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecc000f960 con 0x7fecd00cdbf0 2024-04-25T01:11:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecd00cb030 con 0x7fecd00cdbf0 2024-04-25T01:11:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecd0069040 con 0x7fecd00cdbf0 2024-04-25T01:11:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecb77fe640 1 -- 172.21.15.33:0/270411871 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fecc000fb00 con 0x7fecd00cdbf0 2024-04-25T01:11:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecb77fe640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fecbc0678c0 0x7fecbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:11:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecb77fe640 1 -- 172.21.15.33:0/270411871 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fecc0085ea0 con 0x7fecd00cdbf0 2024-04-25T01:11:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.790+0000 7fecceffd640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fecbc0678c0 0x7fecbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:11:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.794+0000 7fecceffd640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fecbc0678c0 0x7fecbc069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c10 tx=0x7fecc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:11:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.794+0000 7fecb77fe640 1 -- 172.21.15.33:0/270411871 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fecc0019030 con 0x7fecd00cdbf0 2024-04-25T01:11:59.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.954+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fecd008dbe0 con 0x7fecbc0678c0 2024-04-25T01:11:59.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecb77fe640 1 -- 172.21.15.33:0/270411871 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fecd008dbe0 con 0x7fecbc0678c0 2024-04-25T01:11:59.949 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T01:11:59.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fecbc0678c0 msgr2=0x7fecbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fecbc0678c0 0x7fecbc069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fecc4000c10 tx=0x7fecc4009040 comp rx=0 tx=0).stop 2024-04-25T01:11:59.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecd00cdbf0 msgr2=0x7fecd01f4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:11:59.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecd00cdbf0 0x7fecd01f4610 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fecc0011630 tx=0x7fecc0011af0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 shutdown_connections 2024-04-25T01:11:59.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fecbc0678c0 0x7fecbc069d80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecd006cc60 0x7fecd006d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecd00cdbf0 0x7fecd01f4610 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 --2- 172.21.15.33:0/270411871 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fecb8004d50 0x7fecd00cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:11:59.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 >> 172.21.15.33:0/270411871 conn(0x7fecd00d3e30 msgr2=0x7fecd00d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:11:59.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 shutdown_connections 2024-04-25T01:11:59.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:11:59.958+0000 7fecd5554640 1 -- 172.21.15.33:0/270411871 wait complete. 2024-04-25T01:11:59.967 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T01:11:59.968 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T01:12:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bd7020640 1 Processor -- start 2024-04-25T01:12:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bd7020640 1 -- start start 2024-04-25T01:12:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bd7020640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f6bd00d0320 0x7f6bd006e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bd7020640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd00b2b40 con 0x7f6bd01457d0 2024-04-25T01:12:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bd7020640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bd00b2de0 con 0x7f6bd01453f0 2024-04-25T01:12:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bd7020640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd00b3430 con 0x7f6bd00d0320 2024-04-25T01:12:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bcffff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6bd01457d0 0x7f6bd006cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40508/0 (socket says 172.21.15.33:40508) 2024-04-25T01:12:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bcffff640 1 -- 172.21.15.33:0/1897199813 learned_addr learned my addr 172.21.15.33:0/1897199813 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:12:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.106+0000 7f6bd55bd640 1 --2- 172.21.15.33:0/1897199813 >> v2:172.21.15.33:3300/0 conn(0x7f6bd00d0320 0x7f6bd006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bcf7fe640 1 -- 172.21.15.33:0/1897199813 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4054387999 0 0) 0x7f6bd00b2b40 con 0x7f6bd01457d0 2024-04-25T01:12:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bcf7fe640 1 -- 172.21.15.33:0/1897199813 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6bb8003530 con 0x7f6bd01457d0 2024-04-25T01:12:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd55bd640 1 -- 172.21.15.33:0/1897199813 >> v1:172.21.15.138:6789/0 conn(0x7f6bd01453f0 legacy=0x7f6bd00cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd55bd640 1 -- 172.21.15.33:0/1897199813 >> v1:172.21.15.33:6789/0 conn(0x7f6bd01457d0 legacy=0x7f6bd006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd55bd640 1 -- 172.21.15.33:0/1897199813 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd00b30c0 con 0x7f6bd00d0320 2024-04-25T01:12:00.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd55bd640 1 --2- 172.21.15.33:0/1897199813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd00d0320 0x7f6bd006e870 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00097c0 tx=0x7f6bc001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d0bb6cdc400426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bcf7fe640 1 -- 172.21.15.33:0/1897199813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00b1070 con 0x7f6bd00d0320 2024-04-25T01:12:00.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bcf7fe640 1 -- 172.21.15.33:0/1897199813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc00ac410 con 0x7f6bd00d0320 2024-04-25T01:12:00.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bcf7fe640 1 -- 172.21.15.33:0/1897199813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc00ac7e0 con 0x7f6bd00d0320 2024-04-25T01:12:00.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd7020640 1 -- 172.21.15.33:0/1897199813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd00d0320 msgr2=0x7f6bd006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd7020640 1 --2- 172.21.15.33:0/1897199813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd00d0320 0x7f6bd006e870 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6bc00097c0 tx=0x7f6bc001c4a0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd7020640 1 -- 172.21.15.33:0/1897199813 shutdown_connections 2024-04-25T01:12:00.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd7020640 1 --2- 172.21.15.33:0/1897199813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd00d0320 0x7f6bd006e870 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd7020640 1 -- 172.21.15.33:0/1897199813 >> 172.21.15.33:0/1897199813 conn(0x7f6bd006b3e0 msgr2=0x7f6bd006b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:00.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd7020640 1 -- 172.21.15.33:0/1897199813 shutdown_connections 2024-04-25T01:12:00.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.110+0000 7f6bd7020640 1 -- 172.21.15.33:0/1897199813 wait complete. 2024-04-25T01:12:00.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 Processor -- start 2024-04-25T01:12:00.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 -- start start 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd00d0320 0x7f6bd00b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd01453f0 0x7f6bd0070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bd01457d0 0x7f6bd0159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd016a070 con 0x7f6bd01453f0 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd0169ef0 con 0x7f6bd00d0320 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd016a1f0 con 0x7f6bd01457d0 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd01453f0 0x7f6bd0070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd01453f0 0x7f6bd0070cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49752/0 (socket says 172.21.15.33:49752) 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 -- 172.21.15.33:0/985718928 learned_addr learned my addr 172.21.15.33:0/985718928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd4dbc640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd00d0320 0x7f6bd00b0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd55bd640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bd01457d0 0x7f6bd0159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 -- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bd01457d0 msgr2=0x7f6bd0159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bd01457d0 0x7f6bd0159750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 -- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd00d0320 msgr2=0x7f6bd00b0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6bd00d0320 0x7f6bd00b0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 -- 172.21.15.33:0/985718928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd0071410 con 0x7f6bd01453f0 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd55bd640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6bd01457d0 0x7f6bd0159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcffff640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd01453f0 0x7f6bd0070cc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400c420 tx=0x7f6bc400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4017070 con 0x7f6bd01453f0 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bc4005cf0 con 0x7f6bd01453f0 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc40072d0 con 0x7f6bd01453f0 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd00716f0 con 0x7f6bd01453f0 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd0071cb0 con 0x7f6bd01453f0 2024-04-25T01:12:00.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bd0072090 con 0x7f6bd01453f0 2024-04-25T01:12:00.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.114+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6bc4007470 con 0x7f6bd01453f0 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bcd7fa640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6bbc0678c0 0x7f6bbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f6bc4085860 con 0x7f6bd01453f0 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bcd7fa640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7f6bbc06dd60 0x7f6bbc0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 --> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6bc4018e50 con 0x7f6bbc06dd60 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f6bc4085cf0 con 0x7f6bd01453f0 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bd4dbc640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6bbc0678c0 0x7f6bbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bd55bd640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7f6bbc06dd60 0x7f6bbc0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bd4dbc640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6bbc0678c0 0x7f6bbc069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00b0460 tx=0x7f6bc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bd55bd640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7f6bbc06dd60 0x7f6bbc0701a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.118+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== osd.7 v2:172.21.15.138:6800/400031455 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6bc4018e50 con 0x7f6bbc06dd60 2024-04-25T01:12:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 --> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6bd0069040 con 0x7f6bbc06dd60 2024-04-25T01:12:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bcd7fa640 1 -- 172.21.15.33:0/985718928 <== osd.7 v2:172.21.15.138:6800/400031455 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6bd0069040 con 0x7f6bbc06dd60 2024-04-25T01:12:00.125 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:12:00.125 INFO:teuthology.orchestra.run.smithi033.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T01:12:00.125 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:12:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7f6bbc06dd60 msgr2=0x7f6bbc0701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6800/400031455,v1:172.21.15.138:6802/400031455] conn(0x7f6bbc06dd60 0x7f6bbc0701a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6bbc0678c0 msgr2=0x7f6bbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6bbc0678c0 0x7f6bbc069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6bd00b0460 tx=0x7f6bc8009040 comp rx=0 tx=0).stop 2024-04-25T01:12:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd01453f0 msgr2=0x7f6bd0070cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 --2- 172.21.15.33:0/985718928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6bd01453f0 0x7f6bd0070cc0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400c420 tx=0x7f6bc400c8e0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd55bd640 1 -- 172.21.15.33:0/985718928 reap_dead start 2024-04-25T01:12:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 shutdown_connections 2024-04-25T01:12:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 >> 172.21.15.33:0/985718928 conn(0x7f6bd006b3e0 msgr2=0x7f6bd015e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 shutdown_connections 2024-04-25T01:12:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.134+0000 7f6bd7020640 1 -- 172.21.15.33:0/985718928 wait complete. 2024-04-25T01:12:00.141 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T01:12:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd55d79f640 1 Processor -- start 2024-04-25T01:12:00.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd55d79f640 1 -- start start 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd55d79f640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd5580b4c50 0x7fd55815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd55d79f640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5580b5680 0x7fd5580b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd55d79f640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd55806f020 con 0x7fd5581563d0 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd55d79f640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fd55806f2a0 con 0x7fd5580b5680 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd55d79f640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fd55806f580 con 0x7fd5580b4c50 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5577fe640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fd5580b5680 0x7fd5580b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd556ffd640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fd5581563d0 0x7fd55815cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40520/0 (socket says 172.21.15.33:40520) 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd556ffd640 1 -- 172.21.15.33:0/2903831214 learned_addr learned my addr 172.21.15.33:0/2903831214 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5577fe640 1 --2- 172.21.15.33:0/2903831214 >> v2:172.21.15.33:3300/0 conn(0x7fd5580b5680 0x7fd5580b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49756/0 (socket says 172.21.15.33:49756) 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5567fc640 1 --2- 172.21.15.33:0/2903831214 >> v2:172.21.15.138:3300/0 conn(0x7fd5580b4c50 0x7fd55815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5577fe640 1 -- 172.21.15.33:0/2903831214 >> v2:172.21.15.138:3300/0 conn(0x7fd5580b4c50 msgr2=0x7fd55815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5577fe640 1 --2- 172.21.15.33:0/2903831214 >> v2:172.21.15.138:3300/0 conn(0x7fd5580b4c50 0x7fd55815d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5577fe640 1 -- 172.21.15.33:0/2903831214 >> v1:172.21.15.33:6789/0 conn(0x7fd5581563d0 legacy=0x7fd55815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd555ffb640 1 -- 172.21.15.33:0/2903831214 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019105233 0 0) 0x7fd548001120 con 0x7fd5581563d0 2024-04-25T01:12:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5577fe640 1 -- 172.21.15.33:0/2903831214 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55806f860 con 0x7fd5580b5680 2024-04-25T01:12:00.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.266+0000 7fd5567fc640 1 --2- 172.21.15.33:0/2903831214 >> v2:172.21.15.138:3300/0 conn(0x7fd5580b4c50 0x7fd55815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:12:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd5577fe640 1 --2- 172.21.15.33:0/2903831214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5580b5680 0x7fd5580b5aa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd5580b69f0 tx=0x7fd55001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccd99e0c92e664d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd555ffb640 1 -- 172.21.15.33:0/2903831214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500b0070 con 0x7fd5580b5680 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd555ffb640 1 -- 172.21.15.33:0/2903831214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5500ab600 con 0x7fd5580b5680 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd555ffb640 1 -- 172.21.15.33:0/2903831214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500ab9d0 con 0x7fd5580b5680 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 -- 172.21.15.33:0/2903831214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5580b5680 msgr2=0x7fd5580b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 --2- 172.21.15.33:0/2903831214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5580b5680 0x7fd5580b5aa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd5580b69f0 tx=0x7fd55001c3f0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 -- 172.21.15.33:0/2903831214 shutdown_connections 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 --2- 172.21.15.33:0/2903831214 >> v2:172.21.15.138:3300/0 conn(0x7fd5580b4c50 0x7fd55815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 --2- 172.21.15.33:0/2903831214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5580b5680 0x7fd5580b5aa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 -- 172.21.15.33:0/2903831214 >> 172.21.15.33:0/2903831214 conn(0x7fd5580734c0 msgr2=0x7fd55809a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 -- 172.21.15.33:0/2903831214 shutdown_connections 2024-04-25T01:12:00.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 -- 172.21.15.33:0/2903831214 wait complete. 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 Processor -- start 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.270+0000 7fd55d79f640 1 -- start start 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5580b4c50 0x7fd558148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5580b5680 0x7fd5581568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5581563d0 0x7fd558156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd558070ee0 con 0x7fd5581563d0 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd558070d60 con 0x7fd5580b4c50 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fd558071060 con 0x7fd5580b5680 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5581563d0 0x7fd558156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5581563d0 0x7fd558156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49760/0 (socket says 172.21.15.33:49760) 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 -- 172.21.15.33:0/3790796603 learned_addr learned my addr 172.21.15.33:0/3790796603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:12:00.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd556ffd640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5580b4c50 0x7fd558148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5567fc640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5580b5680 0x7fd5581568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 -- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5580b5680 msgr2=0x7fd5581568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5580b5680 0x7fd5581568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 -- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5580b4c50 msgr2=0x7fd558148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5580b4c50 0x7fd558148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 -- 172.21.15.33:0/3790796603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd558158500 con 0x7fd5581563d0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5567fc640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5580b5680 0x7fd5581568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd5577fe640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5581563d0 0x7fd558156e30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ae040 tx=0x7fd55001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd53f7fe640 1 -- 172.21.15.33:0/3790796603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500b0050 con 0x7fd5581563d0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd53f7fe640 1 -- 172.21.15.33:0/3790796603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5500bc070 con 0x7fd5581563d0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd558151b70 con 0x7fd5581563d0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd53f7fe640 1 -- 172.21.15.33:0/3790796603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500b5870 con 0x7fd5581563d0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd558157740 con 0x7fd5581563d0 2024-04-25T01:12:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd558069040 con 0x7fd5581563d0 2024-04-25T01:12:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd53f7fe640 1 -- 172.21.15.33:0/3790796603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd5500b5a10 con 0x7fd5581563d0 2024-04-25T01:12:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd53f7fe640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd5440678c0 0x7fd544069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd53f7fe640 1 -- 172.21.15.33:0/3790796603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7fd550120450 con 0x7fd5581563d0 2024-04-25T01:12:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd556ffd640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd5440678c0 0x7fd544069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.274+0000 7fd556ffd640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd5440678c0 0x7fd544069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd548007fd0 tx=0x7fd548007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.278+0000 7fd53f7fe640 1 -- 172.21.15.33:0/3790796603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7fd550016070 con 0x7fd5581563d0 2024-04-25T01:12:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.434+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fd558073d40 con 0x7fd5440678c0 2024-04-25T01:12:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.434+0000 7fd53f7fe640 1 -- 172.21.15.33:0/3790796603 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd558073d40 con 0x7fd5440678c0 2024-04-25T01:12:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T01:12:00.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd5440678c0 msgr2=0x7fd544069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd5440678c0 0x7fd544069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd548007fd0 tx=0x7fd548007980 comp rx=0 tx=0).stop 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5581563d0 msgr2=0x7fd558156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5581563d0 0x7fd558156e30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd5500ae040 tx=0x7fd55001ac30 comp rx=0 tx=0).stop 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 shutdown_connections 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fd5440678c0 0x7fd544069d80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5581563d0 0x7fd558156e30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fd5580b5680 0x7fd5581568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 --2- 172.21.15.33:0/3790796603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5580b4c50 0x7fd558148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 >> 172.21.15.33:0/3790796603 conn(0x7fd5580734c0 msgr2=0x7fd5581557a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 shutdown_connections 2024-04-25T01:12:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.438+0000 7fd55d79f640 1 -- 172.21.15.33:0/3790796603 wait complete. 2024-04-25T01:12:00.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:12:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.574+0000 7f39d8ffd640 1 Processor -- start 2024-04-25T01:12:00.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.574+0000 7f39d8ffd640 1 -- start start 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39d415e660 0x7f39d4071e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d4072370 0x7f39d40ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39d41448d0 con 0x7f39d41444f0 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7f39d40ce0c0 con 0x7f39d415e660 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39d415ec20 con 0x7f39d4072370 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39c9d9b640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7f39d415e660 0x7f39d4071e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d259c640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f39d41444f0 0x7f39d40b5620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:40528/0 (socket says 172.21.15.33:40528) 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d259c640 1 -- 172.21.15.33:0/1953118150 learned_addr learned my addr 172.21.15.33:0/1953118150 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39c9d9b640 1 -- 172.21.15.33:0/1953118150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d4072370 msgr2=0x7f39d40ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d2d9d640 1 --2- 172.21.15.33:0/1953118150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d4072370 0x7f39d40ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39c9d9b640 1 --2- 172.21.15.33:0/1953118150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d4072370 0x7f39d40ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d1d9b640 1 -- 172.21.15.33:0/1953118150 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2151216378 0 0) 0x7f39c4001120 con 0x7f39d41444f0 2024-04-25T01:12:00.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39c9d9b640 1 -- 172.21.15.33:0/1953118150 >> v1:172.21.15.33:6789/0 conn(0x7f39d41444f0 legacy=0x7f39d40b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39c9d9b640 1 -- 172.21.15.33:0/1953118150 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d415ef00 con 0x7f39d415e660 2024-04-25T01:12:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d2d9d640 1 --2- 172.21.15.33:0/1953118150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d4072370 0x7f39d40ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:12:00.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39c9d9b640 1 --2- 172.21.15.33:0/1953118150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d415e660 0x7f39d4071e30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0097c0 tx=0x7f39bc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28ca30a941844df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d1d9b640 1 -- 172.21.15.33:0/1953118150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0b0070 con 0x7f39d415e660 2024-04-25T01:12:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d1d9b640 1 -- 172.21.15.33:0/1953118150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39bc0ab600 con 0x7f39d415e660 2024-04-25T01:12:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d1d9b640 1 -- 172.21.15.33:0/1953118150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0ab9d0 con 0x7f39d415e660 2024-04-25T01:12:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1953118150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d415e660 msgr2=0x7f39d4071e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1953118150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d415e660 0x7f39d4071e30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0097c0 tx=0x7f39bc01c4a0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1953118150 shutdown_connections 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1953118150 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d4072370 0x7f39d40ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1953118150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d415e660 0x7f39d4071e30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1953118150 >> 172.21.15.33:0/1953118150 conn(0x7f39d40b1810 msgr2=0x7f39d40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1953118150 shutdown_connections 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.578+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1953118150 wait complete. 2024-04-25T01:12:00.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 Processor -- start 2024-04-25T01:12:00.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 -- start start 2024-04-25T01:12:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d4072370 0x7f39d4158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d41444f0 0x7f39d4138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d415e660 0x7f39d40b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4167a00 con 0x7f39d4072370 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f39d4072780 con 0x7f39d41444f0 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f39d4167b80 con 0x7f39d415e660 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d4072370 0x7f39d4158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d4072370 0x7f39d4158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49778/0 (socket says 172.21.15.33:49778) 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 -- 172.21.15.33:0/1608253235 learned_addr learned my addr 172.21.15.33:0/1608253235 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39c9d9b640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d41444f0 0x7f39d4138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d2d9d640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d415e660 0x7f39d40b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 -- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d415e660 msgr2=0x7f39d40b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d415e660 0x7f39d40b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 -- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d41444f0 msgr2=0x7f39d4138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d41444f0 0x7f39d4138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 -- 172.21.15.33:0/1608253235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4139320 con 0x7f39d4072370 2024-04-25T01:12:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d259c640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d4072370 0x7f39d4158e20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f39c400c7b0 tx=0x7f39c400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39cb7fe640 1 -- 172.21.15.33:0/1608253235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4018070 con 0x7f39d4072370 2024-04-25T01:12:00.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39cb7fe640 1 -- 172.21.15.33:0/1608253235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39c40072f0 con 0x7f39d4072370 2024-04-25T01:12:00.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39cb7fe640 1 -- 172.21.15.33:0/1608253235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c40047f0 con 0x7f39d4072370 2024-04-25T01:12:00.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d4139600 con 0x7f39d4072370 2024-04-25T01:12:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d4139bc0 con 0x7f39d4072370 2024-04-25T01:12:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.582+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4069040 con 0x7f39d4072370 2024-04-25T01:12:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.586+0000 7f39cb7fe640 1 -- 172.21.15.33:0/1608253235 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f39c4004990 con 0x7f39d4072370 2024-04-25T01:12:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.586+0000 7f39cb7fe640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f39b40678f0 0x7f39b4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.586+0000 7f39cb7fe640 1 -- 172.21.15.33:0/1608253235 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f39c4084ed0 con 0x7f39d4072370 2024-04-25T01:12:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.586+0000 7f39c9d9b640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f39b40678f0 0x7f39b4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:00.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.586+0000 7f39c9d9b640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f39b40678f0 0x7f39b4069db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0ad000 tx=0x7f39bc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:00.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.590+0000 7f39cb7fe640 1 -- 172.21.15.33:0/1608253235 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f39d41448d0 con 0x7f39d4072370 2024-04-25T01:12:00.592 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:12:00.595 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:12:00.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.750+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39d40b2f20 con 0x7f39b40678f0 2024-04-25T01:12:00.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.750+0000 7f39cb7fe640 1 -- 172.21.15.33:0/1608253235 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13404 (secure 0 0 0) 0x7f39d40b2f20 con 0x7f39b40678f0 2024-04-25T01:12:00.742 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:12:00.743 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f39b40678f0 msgr2=0x7f39b4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f39b40678f0 0x7f39b4069db0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0ad000 tx=0x7f39bc0a6000 comp rx=0 tx=0).stop 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d4072370 msgr2=0x7f39d4158e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d4072370 0x7f39d4158e20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f39c400c7b0 tx=0x7f39c400cc70 comp rx=0 tx=0).stop 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 shutdown_connections 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f39b40678f0 0x7f39b4069db0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f39d415e660 0x7f39d40b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39d41444f0 0x7f39d4138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 --2- 172.21.15.33:0/1608253235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d4072370 0x7f39d4158e20 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:00.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 >> 172.21.15.33:0/1608253235 conn(0x7f39d40b1810 msgr2=0x7f39d406d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 shutdown_connections 2024-04-25T01:12:00.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:00.754+0000 7f39d8ffd640 1 -- 172.21.15.33:0/1608253235 wait complete. 2024-04-25T01:12:00.762 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-04-25T01:11:59.164562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229660,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745060,"statfs":{"total":773094113280,"available":772858941440,"internally_reserved":0,"allocated":5283840,"data_stored":2971392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-7834,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002293"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T01:11:52.781200+0000","last_change":"2024-04-25T01:08:18.970463+0000","last_active":"2024-04-25T01:11:52.781200+0000","last_peered":"2024-04-25T01:11:52.781200+0000","last_clean":"2024-04-25T01:11:52.781200+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:11:52.781200+0000","last_undegraded":"2024-04-25T01:11:52.781200+0000","last_fullsized":"2024-04-25T01:11:52.781200+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:08:16.134265+0000","last_clean_scrub_stamp":"2024-04-25T01:08:16.134265+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:02:10.098427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509487,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1105920,"data_stored":814200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509486,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509488,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339576,"statfs":{"total":96636764160,"available":96603725824,"internally_reserved":0,"allocated":1105920,"data_stored":814200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:12:00.765 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T01:08:16.134265+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=8, tm_sec=16, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=1, tm_min=11, tm_sec=55, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T01:12:00.765 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T01:12:20.766 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T01:12:20.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc27c0640 1 Processor -- start 2024-04-25T01:12:20.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc27c0640 1 -- start start 2024-04-25T01:12:20.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc27c0640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6fbc06e1c0 0x7f6fbc0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc27c0640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fbc14bf30 con 0x7f6fbc070d20 2024-04-25T01:12:20.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc27c0640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fbc071900 con 0x7f6fbc14bb50 2024-04-25T01:12:20.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc27c0640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f6fbc072d40 con 0x7f6fbc06e1c0 2024-04-25T01:12:20.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc0d5d640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f6fbc070d20 0x7f6fbc0b3560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59146/0 (socket says 172.21.15.33:59146) 2024-04-25T01:12:20.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc0d5d640 1 -- 172.21.15.33:0/655260479 learned_addr learned my addr 172.21.15.33:0/655260479 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:12:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbb7fe640 1 --2- 172.21.15.33:0/655260479 >> v2:172.21.15.138:3300/0 conn(0x7f6fbc06e1c0 0x7f6fbc0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2260415764 0 0) 0x7f6fbc14bf30 con 0x7f6fbc070d20 2024-04-25T01:12:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6fa8003530 con 0x7f6fbc070d20 2024-04-25T01:12:20.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2173407636 0 0) 0x7f6fa8003530 con 0x7f6fbc070d20 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 >> v2:172.21.15.138:3300/0 conn(0x7f6fbc06e1c0 msgr2=0x7f6fbc0cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 --2- 172.21.15.33:0/655260479 >> v2:172.21.15.138:3300/0 conn(0x7f6fbc06e1c0 0x7f6fbc0cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 >> v1:172.21.15.138:6789/0 conn(0x7f6fbc14bb50 legacy=0x7f6fbc06dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc071be0 con 0x7f6fbc070d20 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f6fb4001680 con 0x7f6fbc070d20 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 >> v1:172.21.15.33:6789/0 conn(0x7f6fbc070d20 legacy=0x7f6fbc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa80039c0 0x7f6fa8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa8004810 0x7f6fa8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fa80051d0 0x7f6fa800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa800c040 con 0x7f6fa8004810 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa800c1f0 con 0x7f6fa80039c0 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa800c3f0 con 0x7f6fa80051d0 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbb7fe640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa8004810 0x7f6fa8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:20.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbbfff640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa80039c0 0x7f6fa8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbb7fe640 1 -- 172.21.15.33:0/655260479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fa80051d0 msgr2=0x7f6fa800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbb7fe640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fa80051d0 0x7f6fa800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbb7fe640 1 -- 172.21.15.33:0/655260479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa80039c0 msgr2=0x7f6fa8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbb7fe640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa80039c0 0x7f6fa8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fbb7fe640 1 -- 172.21.15.33:0/655260479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa8003530 con 0x7f6fa8004810 2024-04-25T01:12:20.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.914+0000 7f6fc0d5d640 1 -- 172.21.15.33:0/655260479 reap_dead start 2024-04-25T01:12:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fbb7fe640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa8004810 0x7f6fa8004c90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009ae0 tx=0x7f6fa401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c8a538868d054b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40b2070 con 0x7f6fa8004810 2024-04-25T01:12:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa40a5660 con 0x7f6fa8004810 2024-04-25T01:12:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fbaffd640 1 -- 172.21.15.33:0/655260479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa40096b0 con 0x7f6fa8004810 2024-04-25T01:12:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 -- 172.21.15.33:0/655260479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa8004810 msgr2=0x7f6fa8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa8004810 0x7f6fa8004c90 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009ae0 tx=0x7f6fa401ca70 comp rx=0 tx=0).stop 2024-04-25T01:12:20.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 -- 172.21.15.33:0/655260479 shutdown_connections 2024-04-25T01:12:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/655260479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fa8004810 0x7f6fa8004c90 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 -- 172.21.15.33:0/655260479 >> 172.21.15.33:0/655260479 conn(0x7f6fbc0b1810 msgr2=0x7f6fbc0b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 -- 172.21.15.33:0/655260479 shutdown_connections 2024-04-25T01:12:20.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 -- 172.21.15.33:0/655260479 wait complete. 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 Processor -- start 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.918+0000 7f6fc27c0640 1 -- start start 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa8004810 0x7f6fbc141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc14a480 0x7f6fbc1ee340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fbc15f030 0x7f6fbc15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0b3040 con 0x7f6fbc15f030 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0b2b10 con 0x7f6fa8004810 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc0cce40 con 0x7f6fbc14a480 2024-04-25T01:12:20.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fbc15f030 0x7f6fbc15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:20.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fbc15f030 0x7f6fbc15f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55950/0 (socket says 172.21.15.33:55950) 2024-04-25T01:12:20.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 -- 172.21.15.33:0/2697734758 learned_addr learned my addr 172.21.15.33:0/2697734758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:12:20.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fbbfff640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa8004810 0x7f6fbc141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:20.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fbb7fe640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc14a480 0x7f6fbc1ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:20.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 -- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc14a480 msgr2=0x7f6fbc1ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:20.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc14a480 0x7f6fbc1ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 -- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa8004810 msgr2=0x7f6fbc141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa8004810 0x7f6fbc141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 -- 172.21.15.33:0/2697734758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc14b200 con 0x7f6fbc15f030 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fbb7fe640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc14a480 0x7f6fbc1ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fbbfff640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa8004810 0x7f6fbc141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc0d5d640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fbc15f030 0x7f6fbc15f4c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc06e610 tx=0x7f6fb400f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fb8ff9640 1 -- 172.21.15.33:0/2697734758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400fea0 con 0x7f6fbc15f030 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fb8ff9640 1 -- 172.21.15.33:0/2697734758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb4007d00 con 0x7f6fbc15f030 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fb8ff9640 1 -- 172.21.15.33:0/2697734758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400a290 con 0x7f6fbc15f030 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc14a960 con 0x7f6fbc15f030 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc14af20 con 0x7f6fbc15f030 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc069040 con 0x7f6fbc15f030 2024-04-25T01:12:20.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fb8ff9640 1 -- 172.21.15.33:0/2697734758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6fb400a430 con 0x7f6fbc15f030 2024-04-25T01:12:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fb8ff9640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f9c0678c0 0x7f6f9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fb8ff9640 1 -- 172.21.15.33:0/2697734758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f6fb4088020 con 0x7f6fbc15f030 2024-04-25T01:12:20.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.922+0000 7f6fbbfff640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f9c0678c0 0x7f6f9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:20.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.926+0000 7f6fbbfff640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f9c0678c0 0x7f6f9c069d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fac000f90 tx=0x7f6fac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:20.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:20.926+0000 7f6fb8ff9640 1 -- 172.21.15.33:0/2697734758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f6fb4058d00 con 0x7f6fbc15f030 2024-04-25T01:12:21.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.086+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 --> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6fbc142180 con 0x7f6f9c0678c0 2024-04-25T01:12:21.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.090+0000 7f6fb8ff9640 1 -- 172.21.15.33:0/2697734758 <== mgr.4105 v2:172.21.15.138:6832/2108469693 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7f6fbc142180 con 0x7f6f9c0678c0 2024-04-25T01:12:21.082 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:12:21.082 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f9c0678c0 msgr2=0x7f6f9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f9c0678c0 0x7f6f9c069d80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fac000f90 tx=0x7f6fac009040 comp rx=0 tx=0).stop 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fbc15f030 msgr2=0x7f6fbc15f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fbc15f030 0x7f6fbc15f4c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc06e610 tx=0x7f6fb400f820 comp rx=0 tx=0).stop 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 shutdown_connections 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f6f9c0678c0 0x7f6f9c069d80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fbc15f030 0x7f6fbc15f4c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f6fbc14a480 0x7f6fbc1ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 --2- 172.21.15.33:0/2697734758 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fa8004810 0x7f6fbc141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 >> 172.21.15.33:0/2697734758 conn(0x7f6fbc0b1810 msgr2=0x7f6fbc0cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 shutdown_connections 2024-04-25T01:12:21.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.094+0000 7f6fc27c0640 1 -- 172.21.15.33:0/2697734758 wait complete. 2024-04-25T01:12:21.101 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-04-25T01:12:19.170081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":229644,"kb_used_data":5160,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745076,"statfs":{"total":773094113280,"available":772858957824,"internally_reserved":0,"allocated":5283840,"data_stored":2971392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003386"},"pg_stats":[{"pgid":"1.0","version":"44'119","reported_seq":223,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-25T01:12:00.897739+0000","last_change":"2024-04-25T01:12:00.897739+0000","last_active":"2024-04-25T01:12:00.897739+0000","last_peered":"2024-04-25T01:12:00.897739+0000","last_clean":"2024-04-25T01:12:00.897739+0000","last_became_active":"2024-04-25T01:08:18.968393+0000","last_became_peered":"2024-04-25T01:08:18.968393+0000","last_unstale":"2024-04-25T01:12:00.897739+0000","last_undegraded":"2024-04-25T01:12:00.897739+0000","last_fullsized":"2024-04-25T01:12:00.897739+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'119","last_scrub_stamp":"2024-04-25T01:12:00.897391+0000","last_deep_scrub":"44'119","last_deep_scrub_stamp":"2024-04-25T01:12:00.897391+0000","last_clean_scrub_stamp":"2024-04-25T01:12:00.897391+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:07:46.086394+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1105920,"data_stored":814200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509493,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31680,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509490,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509492,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27520,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344320,"statfs":{"total":96636764160,"available":96608583680,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27524,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344316,"statfs":{"total":96636764160,"available":96608579584,"internally_reserved":0,"allocated":512000,"data_stored":223832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1105920,"data_stored":814200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:12:21.102 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T01:12:21.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fededdd6640 1 Processor -- start 2024-04-25T01:12:21.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fededdd6640 1 -- start start 2024-04-25T01:12:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fededdd6640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fede806eb60 0x7fede8160070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fededdd6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fede8071170 0x7fede80715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fededdd6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fede80cee10 con 0x7fede8145660 2024-04-25T01:12:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fededdd6640 1 -- --> v2:172.21.15.33:3300/0 -- mon_getmap magic: 0 -- 0x7fede8145a40 con 0x7fede806eb60 2024-04-25T01:12:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fededdd6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7fede8070ce0 con 0x7fede8071170 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede6ffd640 1 --2- >> v2:172.21.15.33:3300/0 conn(0x7fede806eb60 0x7fede8160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede77fe640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7fede8145660 0x7fede815f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59160/0 (socket says 172.21.15.33:59160) 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede77fe640 1 -- 172.21.15.33:0/1299778740 learned_addr learned my addr 172.21.15.33:0/1299778740 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede7fff640 1 --2- 172.21.15.33:0/1299778740 >> v2:172.21.15.138:3300/0 conn(0x7fede8071170 0x7fede80715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede6ffd640 1 -- 172.21.15.33:0/1299778740 >> v2:172.21.15.138:3300/0 conn(0x7fede8071170 msgr2=0x7fede80715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede6ffd640 1 --2- 172.21.15.33:0/1299778740 >> v2:172.21.15.138:3300/0 conn(0x7fede8071170 0x7fede80715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede6ffd640 1 -- 172.21.15.33:0/1299778740 >> v1:172.21.15.33:6789/0 conn(0x7fede8145660 legacy=0x7fede815f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede67fc640 1 -- 172.21.15.33:0/1299778740 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526685742 0 0) 0x7fede80cee10 con 0x7fede8145660 2024-04-25T01:12:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede6ffd640 1 -- 172.21.15.33:0/1299778740 --> v2:172.21.15.33:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede80cf0f0 con 0x7fede806eb60 2024-04-25T01:12:21.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.234+0000 7fede7fff640 1 --2- 172.21.15.33:0/1299778740 >> v2:172.21.15.138:3300/0 conn(0x7fede8071170 0x7fede80715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:12:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fede6ffd640 1 --2- 172.21.15.33:0/1299778740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede806eb60 0x7fede8160070 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fede8068b80 tx=0x7feddc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d51d1c7f815b4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fede67fc640 1 -- 172.21.15.33:0/1299778740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc0ab650 con 0x7fede806eb60 2024-04-25T01:12:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fede67fc640 1 -- 172.21.15.33:0/1299778740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc0b0070 con 0x7fede806eb60 2024-04-25T01:12:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fede67fc640 1 -- 172.21.15.33:0/1299778740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc0aba10 con 0x7fede806eb60 2024-04-25T01:12:21.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- 172.21.15.33:0/1299778740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede806eb60 msgr2=0x7fede8160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 --2- 172.21.15.33:0/1299778740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede806eb60 0x7fede8160070 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fede8068b80 tx=0x7feddc01c450 comp rx=0 tx=0).stop 2024-04-25T01:12:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- 172.21.15.33:0/1299778740 shutdown_connections 2024-04-25T01:12:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 --2- 172.21.15.33:0/1299778740 >> v2:172.21.15.138:3300/0 conn(0x7fede8071170 0x7fede80715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 --2- 172.21.15.33:0/1299778740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede806eb60 0x7fede8160070 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- 172.21.15.33:0/1299778740 >> 172.21.15.33:0/1299778740 conn(0x7fede80b1810 msgr2=0x7fede80b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- 172.21.15.33:0/1299778740 shutdown_connections 2024-04-25T01:12:21.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- 172.21.15.33:0/1299778740 wait complete. 2024-04-25T01:12:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 Processor -- start 2024-04-25T01:12:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- start start 2024-04-25T01:12:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede806eb60 0x7fede815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede8071170 0x7fede815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8145660 0x7fede80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fede8160730 con 0x7fede8071170 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fede81605b0 con 0x7fede806eb60 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fededdd6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7fede8168ca0 con 0x7fede8145660 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fede6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede8071170 0x7fede815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fede6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede8071170 0x7fede815b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55978/0 (socket says 172.21.15.33:55978) 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.238+0000 7fede6ffd640 1 -- 172.21.15.33:0/233525279 learned_addr learned my addr 172.21.15.33:0/233525279 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede7fff640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8145660 0x7fede80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede6ffd640 1 -- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8145660 msgr2=0x7fede80aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede77fe640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede806eb60 0x7fede815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede6ffd640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8145660 0x7fede80aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede6ffd640 1 -- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede806eb60 msgr2=0x7fede815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede6ffd640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede806eb60 0x7fede815e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede6ffd640 1 -- 172.21.15.33:0/233525279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede815d620 con 0x7fede8071170 2024-04-25T01:12:21.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede7fff640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8145660 0x7fede80aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede6ffd640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede8071170 0x7fede815b0b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feddc019060 tx=0x7feddc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc004220 con 0x7fede8071170 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feddc0043c0 con 0x7fede8071170 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede815d900 con 0x7fede8071170 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feddc003820 con 0x7fede8071170 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede815dec0 con 0x7fede8071170 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fede8069040 con 0x7fede8071170 2024-04-25T01:12:21.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feddc0039c0 con 0x7fede8071170 2024-04-25T01:12:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fedcffff640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedd40678c0 0x7fedd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7feddc120540 con 0x7fede8071170 2024-04-25T01:12:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede77fe640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedd40678c0 0x7fedd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.242+0000 7fede77fe640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedd40678c0 0x7fedd4069d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fedd8000f70 tx=0x7fedd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:21.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.246+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7feddc016070 con 0x7fede8071170 2024-04-25T01:12:21.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.410+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fede8072ac0 con 0x7fede8071170 2024-04-25T01:12:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.414+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7feddc0f11f0 con 0x7fede8071170 2024-04-25T01:12:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fedcffff640 1 -- 172.21.15.33:0/233525279 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7feddc0b00c0 con 0x7fede8071170 2024-04-25T01:12:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedd40678c0 msgr2=0x7fedd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedd40678c0 0x7fedd4069d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fedd8000f70 tx=0x7fedd8009040 comp rx=0 tx=0).stop 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede8071170 msgr2=0x7fede815b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede8071170 0x7fede815b0b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7feddc019060 tx=0x7feddc01ac90 comp rx=0 tx=0).stop 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 shutdown_connections 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7fedd40678c0 0x7fedd4069d80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7fede8145660 0x7fede80aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fede8071170 0x7fede815b0b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 --2- 172.21.15.33:0/233525279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fede806eb60 0x7fede815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 >> 172.21.15.33:0/233525279 conn(0x7fede80b1810 msgr2=0x7fede815f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 shutdown_connections 2024-04-25T01:12:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.418+0000 7fededdd6640 1 -- 172.21.15.33:0/233525279 wait complete. 2024-04-25T01:12:21.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T01:12:21.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:12:21.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f31250b6640 1 Processor -- start 2024-04-25T01:12:21.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f31250b6640 1 -- start start 2024-04-25T01:12:21.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f31250b6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f31200b5530 0x7f3120166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f31250b6640 1 -- --> v1:172.21.15.33:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312015f9a0 con 0x7f312015c160 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f31250b6640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f312015fc80 con 0x7f3120145470 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f31250b6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 -- 0x7f312015f750 con 0x7f31200b5530 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 --1- >> v1:172.21.15.33:6789/0 conn(0x7f312015c160 0x7f31200b4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.33:6789/0 says I am v1:172.21.15.33:59164/0 (socket says 172.21.15.33:59164) 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 -- 172.21.15.33:0/3427310008 learned_addr learned my addr 172.21.15.33:0/3427310008 (peer_addr_for_me v1:172.21.15.33:0/0) 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311f59e640 1 --2- 172.21.15.33:0/3427310008 >> v2:172.21.15.138:3300/0 conn(0x7f31200b5530 0x7f3120166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 <== mon.0 v1:172.21.15.33:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 853067394 0 0) 0x7f312015f9a0 con 0x7f312015c160 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 --> v1:172.21.15.33:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f310c003530 con 0x7f312015c160 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 <== mon.0 v1:172.21.15.33:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 810915266 0 0) 0x7f310c003530 con 0x7f312015c160 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 >> v2:172.21.15.138:3300/0 conn(0x7f31200b5530 msgr2=0x7f3120166ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 --2- 172.21.15.33:0/3427310008 >> v2:172.21.15.138:3300/0 conn(0x7f31200b5530 0x7f3120166ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 >> v1:172.21.15.138:6789/0 conn(0x7f3120145470 legacy=0x7f31200cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 --> v1:172.21.15.33:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312015ff60 con 0x7f312015c160 2024-04-25T01:12:21.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 <== mon.0 v1:172.21.15.33:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 266814974 0 0) 0x7f3110001680 con 0x7f312015c160 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 >> v1:172.21.15.33:6789/0 conn(0x7f312015c160 legacy=0x7f31200b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f310c003a10 0x7f310c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310c004810 0x7f310c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c0051d0 0x7f310c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f312015f750 con 0x7f310c004810 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f310c00c0d0 con 0x7f310c0051d0 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f310c00c2d0 con 0x7f310c003a10 2024-04-25T01:12:21.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310c004810 0x7f310c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311ed9d640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f310c003a10 0x7f310c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311f59e640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c0051d0 0x7f310c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 -- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f310c003a10 msgr2=0x7f310c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f310c003a10 0x7f310c003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 -- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c0051d0 msgr2=0x7f310c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T01:12:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c0051d0 0x7f310c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f3117fff640 1 -- 172.21.15.33:0/3427310008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310c003530 con 0x7f310c004810 2024-04-25T01:12:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311f59e640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c0051d0 0x7f310c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:12:21.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.562+0000 7f311f59e640 1 -- 172.21.15.33:0/3427310008 reap_dead start 2024-04-25T01:12:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f3117fff640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310c004810 0x7f310c004c90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f311000c110 tx=0x7f311001f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc5990c6e73160f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3110007c30 con 0x7f310c004810 2024-04-25T01:12:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f31100ac020 con 0x7f310c004810 2024-04-25T01:12:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f311e59c640 1 -- 172.21.15.33:0/3427310008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31100058e0 con 0x7f310c004810 2024-04-25T01:12:21.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- 172.21.15.33:0/3427310008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310c004810 msgr2=0x7f310c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310c004810 0x7f310c004c90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f311000c110 tx=0x7f311001f270 comp rx=0 tx=0).stop 2024-04-25T01:12:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- 172.21.15.33:0/3427310008 shutdown_connections 2024-04-25T01:12:21.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 --2- 172.21.15.33:0/3427310008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f310c004810 0x7f310c004c90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- 172.21.15.33:0/3427310008 >> 172.21.15.33:0/3427310008 conn(0x7f31200b1810 msgr2=0x7f31200b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- 172.21.15.33:0/3427310008 shutdown_connections 2024-04-25T01:12:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- 172.21.15.33:0/3427310008 wait complete. 2024-04-25T01:12:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 Processor -- start 2024-04-25T01:12:21.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- start start 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c004810 0x7f31201401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 --2- >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31200b0310 0x7f31200c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312006d540 0x7f312006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f312006e850 con 0x7f312006d540 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3120145850 con 0x7f310c004810 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.566+0000 7f31250b6640 1 -- --> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] -- mon_getmap magic: 0 -- 0x7f31200b4e20 con 0x7f31200b0310 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312006d540 0x7f312006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312006d540 0x7f312006d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55998/0 (socket says 172.21.15.33:55998) 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 -- 172.21.15.33:0/1744313643 learned_addr learned my addr 172.21.15.33:0/1744313643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311ed9d640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c004810 0x7f31201401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f3117fff640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31200b0310 0x7f31200c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 -- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31200b0310 msgr2=0x7f31200c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31200b0310 0x7f31200c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 -- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c004810 msgr2=0x7f31201401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c004810 0x7f31201401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 -- 172.21.15.33:0/1744313643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200ae930 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f3117fff640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31200b0310 0x7f31200c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311f59e640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312006d540 0x7f312006d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f31200b5af0 tx=0x7f311800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31177fe640 1 -- 172.21.15.33:0/1744313643 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118016420 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31177fe640 1 -- 172.21.15.33:0/1744313643 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f311801b070 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31200aec10 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31177fe640 1 -- 172.21.15.33:0/1744313643 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31180167e0 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31200af1d0 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3120069040 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31177fe640 1 -- 172.21.15.33:0/1744313643 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3118016980 con 0x7f312006d540 2024-04-25T01:12:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31177fe640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f31000678c0 0x7f3100069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:12:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f31177fe640 1 -- 172.21.15.33:0/1744313643 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5335+0+0 (secure 0 0 0) 0x7f3118086c00 con 0x7f312006d540 2024-04-25T01:12:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311ed9d640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f31000678c0 0x7f3100069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:12:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.570+0000 7f311ed9d640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f31000678c0 0x7f3100069d80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c10 tx=0x7f3108009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:12:21.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.574+0000 7f31177fe640 1 -- 172.21.15.33:0/1744313643 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177576 (secure 0 0 0) 0x7f3118025020 con 0x7f312006d540 2024-04-25T01:12:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.762+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3120071420 con 0x7f312006d540 2024-04-25T01:12:21.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31177fe640 1 -- 172.21.15.33:0/1744313643 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3875 (secure 0 0 0) 0x7f3118057780 con 0x7f312006d540 2024-04-25T01:12:21.756 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:12:21.756 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4838,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3228956110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3228956110},{"type":"v1","addr":"172.21.15.138:6835","nonce":3228956110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5020,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/2116493369","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6833","nonce":2116493369},{"type":"v1","addr":"172.21.15.33:6835","nonce":2116493369}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5032,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2610873006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2610873006},{"type":"v1","addr":"172.21.15.138:6837","nonce":2610873006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5832,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6834/2859647730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":2859647730},{"type":"v1","addr":"172.21.15.33:6834","nonce":2859647730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-04-25T01:12:21.757 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:12:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f31000678c0 msgr2=0x7f3100069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f31000678c0 0x7f3100069d80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3108000c10 tx=0x7f3108009040 comp rx=0 tx=0).stop 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312006d540 msgr2=0x7f312006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312006d540 0x7f312006d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f31200b5af0 tx=0x7f311800f520 comp rx=0 tx=0).stop 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 shutdown_connections 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:6832/2108469693,v1:172.21.15.138:6833/2108469693] conn(0x7f31000678c0 0x7f3100069d80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f312006d540 0x7f312006d9c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3301/0,v1:172.21.15.138:6790/0] conn(0x7f31200b0310 0x7f31200c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 --2- 172.21.15.33:0/1744313643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310c004810 0x7f31201401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 >> 172.21.15.33:0/1744313643 conn(0x7f31200b1810 msgr2=0x7f312006ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 shutdown_connections 2024-04-25T01:12:21.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:12:21.766+0000 7f31250b6640 1 -- 172.21.15.33:0/1744313643 wait complete. 2024-04-25T01:12:21.775 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T01:12:21.775 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T01:12:21.775 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T01:12:21.775 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:21.783 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T01:12:21.784 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T01:12:21.784 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:21.836 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T01:12:21.837 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T01:12:21.837 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:21.852 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T01:12:21.853 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T01:12:21.853 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:21.906 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T01:12:21.906 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T01:12:21.906 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T01:12:21.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:21.978 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T01:12:21.979 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T01:12:21.979 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.065 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T01:12:22.065 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T01:12:22.065 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.158 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T01:12:22.158 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T01:12:22.158 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.244 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T01:12:22.245 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T01:12:22.245 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.320 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T01:12:22.320 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T01:12:22.320 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.407 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T01:12:22.407 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T01:12:22.407 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.470 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T01:12:22.471 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T01:12:22.471 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.568 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T01:12:22.568 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T01:12:22.568 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T01:12:22.568 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.596 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T01:12:22.597 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T01:12:22.597 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.625 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T01:12:22.626 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T01:12:22.626 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T01:12:22.626 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.660 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T01:12:22.660 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T01:12:22.660 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.681 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T01:12:22.682 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T01:12:22.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T01:12:22.739 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T01:12:22.739 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T01:12:22.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-04-25T01:12:22.758 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:12:22.758 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T01:12:23.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:12:23.069 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T01:12:23.283 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:12:23.283 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T01:12:23.599 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:12:23.599 DEBUG:teuthology.orchestra.run.smithi033:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T01:12:23.782 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:12:23.783 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T01:12:23.998 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:12:23.998 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T01:12:24.174 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:12:24.175 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T01:12:24.332 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:12:24.332 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T01:12:24.501 INFO:tasks.ceph:Archiving mon data... 2024-04-25T01:12:24.502 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/data/mon.a.tgz 2024-04-25T01:12:24.502 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-04-25T01:12:24.508 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.rIcHVdGkvN 2024-04-25T01:12:24.509 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.rIcHVdGkvN 2024-04-25T01:12:24.637 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.rIcHVdGkvN 2024-04-25T01:12:24.700 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.rIcHVdGkvN is 326KB 2024-04-25T01:12:24.762 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.rIcHVdGkvN 2024-04-25T01:12:24.767 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/data/mon.b.tgz 2024-04-25T01:12:24.768 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-25T01:12:24.773 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.9Bpkkeo1IJ 2024-04-25T01:12:24.774 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.9Bpkkeo1IJ 2024-04-25T01:12:24.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.9Bpkkeo1IJ 2024-04-25T01:12:24.986 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.9Bpkkeo1IJ is 433KB 2024-04-25T01:12:25.052 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.9Bpkkeo1IJ 2024-04-25T01:12:25.058 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/data/mon.c.tgz 2024-04-25T01:12:25.059 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-04-25T01:12:25.106 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.g6iAgGP9Er 2024-04-25T01:12:25.107 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.g6iAgGP9Er 2024-04-25T01:12:25.255 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.g6iAgGP9Er 2024-04-25T01:12:25.318 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.g6iAgGP9Er is 433KB 2024-04-25T01:12:25.380 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.g6iAgGP9Er 2024-04-25T01:12:25.385 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T01:12:25.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T01:12:25.387 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T01:12:25.448 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T01:12:25.491 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T01:12:25.491 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T01:12:25.493 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/remote/smithi033/crash 2024-04-25T01:12:25.494 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T01:12:25.508 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/remote/smithi138/crash 2024-04-25T01:12:25.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T01:12:25.546 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T01:12:25.546 DEBUG:teuthology.orchestra.run.smithi033:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:12:25.553 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T01:12:25.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T01:12:25.652 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T01:12:25.685 INFO:tasks.ceph:Compressing logs... 2024-04-25T01:12:25.685 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:12:25.697 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T01:12:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38772.log 2024-04-25T01:12:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37650.log 2024-04-25T01:12:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38339.log 2024-04-25T01:12:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37650.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38772.log.gz 2024-04-25T01:12:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34385.log 2024-04-25T01:12:25.711 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37650.log.gz 2024-04-25T01:12:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34091.log 2024-04-25T01:12:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38339.log: /var/log/ceph/ceph-client.admin.34385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38616.log 2024-04-25T01:12:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34091.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38339.log.gz 2024-04-25T01:12:25.712 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34385.log.gz 2024-04-25T01:12:25.712 INFO:teuthology.orchestra.run.smithi033.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34091.log.gz 2024-04-25T01:12:25.713 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38391.log 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33800.log 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38616.log.gz 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34402.log 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33800.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38391.log.gz 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3%/var/log/ceph/ceph-client.admin.34402.log: -- replaced with /var/log/ceph/ceph-client.admin.33800.log.gz 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77276.log 2024-04-25T01:12:25.714 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34402.log.gz 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose/var/log/ceph/ceph-mgr.y.log: -- /var/log/ceph/ceph-client.admin.37175.log 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37518.log 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77276.log.gz 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr:gzip/var/log/ceph/ceph-client.admin.37175.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33565.log 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34404.log 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37175.log.gz 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37518.log.gz 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38033.log 2024-04-25T01:12:25.716 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33565.log.gz 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34871.log 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34404.log: /var/log/ceph/ceph-client.admin.38033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37305.log 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34404.log.gz 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38033.log.gz 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34871.log.gz 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36516.log 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37305.log: /var/log/ceph/ceph-client.admin.38313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38287.log 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37305.log.gz 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-04-25T01:12:25.719 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27522.log 2024-04-25T01:12:25.720 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36516.log.gz 2024-04-25T01:12:25.720 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T01:12:25.720 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38287.log: /var/log/ceph/ceph-client.admin.27522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37728.log 2024-04-25T01:12:25.720 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-04-25T01:12:25.726 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27522.log.gz 2024-04-25T01:12:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36223.log 2024-04-25T01:12:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36595.log 2024-04-25T01:12:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38508.log 2024-04-25T01:12:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36223.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37728.log.gz 2024-04-25T01:12:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36595.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T01:12:25.726 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34608.log 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36223.log.gz 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77781.log 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34975.log 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38508.log.gz 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34608.log.gz 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35702.log 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77781.log: /var/log/ceph/ceph-client.admin.34975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33987.log 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34975.log.gz 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77781.log.gz 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34192.log 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35702.log.gz 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27746.log 2024-04-25T01:12:25.727 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33987.log: /var/log/ceph/ceph-client.admin.34192.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33987.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78015.log 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34192.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35157.log 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27746.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36093.log 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78015.log: /var/log/ceph/ceph-client.admin.35157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35105.log 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.78015.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35157.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78171.log 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36093.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5/var/log/ceph/ceph-client.admin.35105.log: --verbose -- /var/log/ceph/ceph-client.admin.32995.log 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78171.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35105.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38235.log 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78171.log.gz 2024-04-25T01:12:25.728 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37832.log 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:gzip/var/log/ceph/ceph-client.admin.38235.log: 89.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.32995.log.gz --verbose 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr: -- /var/log/ceph/ceph-client.admin.38694.log 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37832.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38235.log.gz 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34197.log 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37832.log.gz 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37544.log 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37754.log 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34197.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37544.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34197.log.gz 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36490.log 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37544.log.gz 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78223.log 2024-04-25T01:12:25.737 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37754.log.gz 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36412.log 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36490.log: /var/log/ceph/ceph-client.admin.78223.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36490.log.gz 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78223.log.gz 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77302.log 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78145.log 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36412.log.gz 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.77302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77484.log 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77302.log.gz 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77729.log 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77484.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78145.log.gz 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37884.log 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.77484.log.gz 2024-04-25T01:12:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34395.log 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37884.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77729.log.gz 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.37884.log.gz 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38417.log 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34819.log 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38417.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34395.log.gz 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34143.log 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38417.log.gz 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35183.log 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34819.log.gz 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34949.log 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38564.log 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34143.log.gz 2024-04-25T01:12:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35183.log: /var/log/ceph/ceph-client.admin.34949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32917.log 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35183.log.gz 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38564.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34949.log.gz 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.35566.log 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38564.log.gz 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32917.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38365.log 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32917.log.gz 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.2.35566.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33086.log 2024-04-25T01:12:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32969.log 2024-04-25T01:12:25.747 INFO:teuthology.orchestra.run.smithi033.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.2.35566.log.gz 2024-04-25T01:12:25.747 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38365.log.gz 2024-04-25T01:12:25.747 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34793.log 2024-04-25T01:12:25.747 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32969.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33086.log.gz 2024-04-25T01:12:25.747 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33506.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32969.log.gz 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33250.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33506.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34793.log.gz 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38590.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35644.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33250.log.gz 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35421.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38590.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36938.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.35421.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.33506.log.gz 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77380.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38590.log.gz 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36067.log 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36938.log: /var/log/ceph/ceph-client.admin.77380.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.35421.log.gz 2024-04-25T01:12:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36938.log.gz 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36067.log: gzip -5 88.2% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.77380.log.gz /var/log/ceph/ceph-client.admin.46049.log 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36275.log 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36067.log.gz 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46049.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37466.log 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46049.log.gz 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33196.log 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37466.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36275.log.gz 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36569.log 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37466.log.gz 2024-04-25T01:12:25.749 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33695.log 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33196.log.gz 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33051.log 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36569.log.gz 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33378.log 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33695.log.gz 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34185.log 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33051.log.gz 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37858.log 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34185.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.33378.log.gz --verbose 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr: -- /var/log/ceph/ceph-client.admin.38059.log 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37858.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34185.log.gz 2024-04-25T01:12:25.750 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34176.log 2024-04-25T01:12:25.758 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27790.log 2024-04-25T01:12:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38059.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34191.log 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37858.log.gz 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38059.log.gz 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34176.log: /var/log/ceph/ceph-client.admin.37936.log 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37962.log 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34176.log.gz 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34191.log.gz 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35728.log 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37962.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37936.log.gz 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33774.log 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37962.log.gz 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77536.log 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35728.log.gz 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34399.log 2024-04-25T01:12:25.760 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32943.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77536.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33774.log.gz 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38878.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77536.log.gz 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34399.log.gz 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32943.log.gz 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.38878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36464.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36990.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36464.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33302.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34609.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36464.log.gz 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33302.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36990.log.gz 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34599.log 2024-04-25T01:12:25.761 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34609.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37336.log 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33302.log.gz 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34609.log.gz 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37227.log 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34599.log.gz 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37253.log 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37336.log: /var/log/ceph/ceph-client.admin.37227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36756.log 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37253.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37336.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37227.log.gz 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33276.log 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1%/var/log/ceph/ceph-client.admin.36756.log: -- replaced with /var/log/ceph/ceph-client.admin.37253.log.gz 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36782.log 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7%/var/log/ceph/ceph-client.admin.33276.log: -- replaced with /var/log/ceph/ceph-client.admin.36756.log.gz 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37096.log 2024-04-25T01:12:25.762 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36782.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33276.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35937.log 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.38878.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36782.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37018.log 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37096.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32743.log 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37018.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35937.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37148.log 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37018.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38007.log 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32743.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37492.log 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38007.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37148.log.gz 2024-04-25T01:12:25.763 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38156.log 2024-04-25T01:12:25.779 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27742.log 2024-04-25T01:12:25.779 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27718.log 2024-04-25T01:12:25.779 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.27790.log: /var/log/ceph/ceph-client.admin.27742.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T01:12:25.779 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.27718.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.27742.log.gz 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27790.log.gz 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.27718.log.gz 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27838.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33183.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.27838.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27838.log.gz 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.33183.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.x.log: gzip 90.8% -- replaced with /var/log/ceph/ceph-client.1.33183.log.gz 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33302.log 2024-04-25T01:12:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.3.33302.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.d.log: 91.5% -- replaced with /var/log/ceph/ceph-client.3.33302.log.gz 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.b.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27814.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27766.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.27814.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27814.log.gz 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27670.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.27766.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27766.log.gz 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27694.log 2024-04-25T01:12:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.27670.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27670.log.gz 2024-04-25T01:12:25.782 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.27694.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27694.log.gz 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37492.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38007.log.gz 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27640.log 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38156.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37492.log.gz 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77354.log 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.27640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77659.log 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38156.log.gz 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27640.log.gz 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35471.log 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77354.log: /var/log/ceph/ceph-client.admin.77659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36015.log 2024-04-25T01:12:25.797 INFO:teuthology.orchestra.run.smithi033.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.77354.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77659.log.gz 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35471.log: 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34194.log 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38261.log 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35471.log.gz/var/log/ceph/ceph-client.admin.36015.log: 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37440.log 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36015.log.gz 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34194.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38668.log 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38261.log: /var/log/ceph/ceph-client.admin.37440.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38261.log.gz 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37440.log.gz 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33748.log 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36730.log 2024-04-25T01:12:25.798 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38668.log.gz 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33748.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36673.log 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36730.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33748.log.gz 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-osd.1.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36730.log.gz 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77406.log 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33909.log 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36673.log: /var/log/ceph/ceph-client.admin.77406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37702.log 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36673.log.gz 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77406.log.gz 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33909.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33909.log.gz 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34585.log 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33617.log 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34585.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37702.log.gz 2024-04-25T01:12:25.799 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37910.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36621.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3%/var/log/ceph/ceph-client.admin.33617.log: -- replaced with /var/log/ceph/ceph-client.admin.34585.log.gz 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37910.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33617.log.gz 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37910.log.gz 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34923.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77703.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr: 89.4%/var/log/ceph/ceph-client.admin.34923.log: -- replaced with /var/log/ceph/ceph-client.admin.36621.log.gz 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35296.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34923.log.gz 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77703.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.77703.log.gz 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35989.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77859.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35989.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35296.log.gz 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36886.log 2024-04-25T01:12:25.800 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33961.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35989.log.gz 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77859.log: /var/log/ceph/ceph-client.admin.36886.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77859.log.gz 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36886.log.gz 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38642.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37596.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33961.log: /var/log/ceph/ceph-client.admin.38642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77581.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33961.log.gz 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38642.log.gz 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37596.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37596.log.gz 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33328.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77885.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33328.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.77581.log.gz 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37070.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33852.log 2024-04-25T01:12:25.801 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77885.log: /var/log/ceph/ceph-client.admin.37070.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.33328.log.gz 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37070.log.gz 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77885.log.gz 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34065.log 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38473.log 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5%/var/log/ceph/ceph-client.admin.34065.log: -- replaced with /var/log/ceph/ceph-client.admin.33852.log.gz 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35079.log 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34065.log.gz 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38473.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38473.log.gz 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35027.log 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37570.log 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35079.log: /var/log/ceph/ceph-client.admin.35027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35963.log 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37570.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35079.log.gz 2024-04-25T01:12:25.802 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35027.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78302.log 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37570.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35963.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35963.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77607.log 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78302.log: /var/log/ceph/ceph-client.admin.77607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37624.log 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph.log: 87.5% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78302.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77607.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37279.log 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37780.log 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr: 86.0%/var/log/ceph/ceph-client.admin.37624.log: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37279.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37624.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37279.log.gz 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37676.log 2024-04-25T01:12:25.803 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78093.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37780.log: /var/log/ceph/ceph-client.admin.37676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36327.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78093.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37780.log.gz 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37676.log.gz 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78067.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78093.log.gz 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37044.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36327.log: /var/log/ceph/ceph-client.admin.78067.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36327.log.gz 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78067.log.gz 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36808.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35131.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37044.log: /var/log/ceph/ceph-client.admin.36808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34601.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35131.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.37044.log.gz 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36808.log.gz 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34598.log 2024-04-25T01:12:25.804 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35131.log.gz 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38182.log 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34601.log: /var/log/ceph/ceph-client.admin.34598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77911.log 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38182.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34601.log.gz 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34598.log.gz 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36119.log 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38182.log.gz 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77755.log 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77911.log: /var/log/ceph/ceph-client.admin.36119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33643.log 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77911.log.gz 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3%/var/log/ceph/ceph-client.admin.77755.log: -- replaced with /var/log/ceph/ceph-client.admin.36119.log.gz 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77755.log.gz 2024-04-25T01:12:25.805 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35053.log 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33222.log 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35053.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34845.log 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33643.log.gz 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35053.log.gz 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33222.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33222.log.gz 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35235.log 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33826.log 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34845.log: /var/log/ceph/ceph-client.admin.35235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77963.log 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34845.log.gz 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33826.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33826.log.gz 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35235.log.gz 2024-04-25T01:12:25.806 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36647.log 2024-04-25T01:12:25.816 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36860.log 2024-04-25T01:12:25.816 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36647.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77963.log.gz 2024-04-25T01:12:25.816 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34897.log 2024-04-25T01:12:25.816 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33935.log 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36647.log.gz 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36860.log: /var/log/ceph/ceph-client.admin.34897.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36860.log.gz 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34897.log.gz 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77807.log 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46075.log 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33935.log: /var/log/ceph/ceph-client.admin.77807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78041.log 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.46075.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33935.log.gz 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77807.log.gz 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32710.log 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46075.log.gz 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34596.log 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78041.log: /var/log/ceph/ceph-client.admin.32710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78252.log 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34596.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78041.log.gz 2024-04-25T01:12:25.817 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32710.log.gz 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34196.log 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34596.log.gz 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36964.log 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78252.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34196.log: /var/log/ceph/ceph-client.admin.34039.log 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36964.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34196.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78252.log.gz 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36964.log.gz 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32891.log 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36912.log 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34039.log.gz 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36386.log: /var/log/ceph/ceph-client.admin.32891.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32891.log.gz 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-04-25T01:12:25.818 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37362.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33170.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2%/var/log/ceph/ceph-client.admin.37362.log: -- replaced with /var/log/ceph/ceph-client.admin.36912.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34401.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37362.log.gz 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33170.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33170.log.gz 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35780.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77328.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33722.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35780.log: /var/log/ceph/ceph-client.admin.77328.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34401.log.gz 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35780.log.gz 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38091.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77328.log.gz 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33722.log: /var/log/ceph/ceph-client.admin.38091.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33722.log.gz 2024-04-25T01:12:25.819 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.38091.log.gz 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr: 88.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37122.log 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35616.log 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr: 88.3%/var/log/ceph/ceph-mds.a.log: -- replaced with /var/log/ceph/ceph-client.admin.37122.log.gz 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36041.log 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77458.log 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35616.log: /var/log/ceph/ceph-client.admin.36041.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35616.log.gz 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36041.log.gz 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38538.log 2024-04-25T01:12:25.820 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77458.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.77458.log.gz 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22713.log 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35858.log 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38538.log: /var/log/ceph/ceph.tmp-client.admin.22713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38538.log.gz 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22713.log.gz 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33023.log 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35858.log: /var/log/ceph/ceph-client.admin.34613.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35858.log.gz 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36171.log 2024-04-25T01:12:25.825 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78276.log 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33023.log: /var/log/ceph/ceph-client.admin.36171.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33023.log.gz 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36171.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32808.log 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.78276.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78276.log.gz 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35676.log 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38208.log 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32808.log: /var/log/ceph/ceph-client.admin.35676.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32808.log.gz 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35676.log.gz 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33669.log 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34117.log 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38208.log: /var/log/ceph/ceph-client.admin.33669.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38208.log.gz 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34013.log 2024-04-25T01:12:25.826 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33669.log.gz 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34117.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34117.log.gz 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36353.log 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35754.log 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34013.log: /var/log/ceph/ceph-client.admin.36353.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34013.log.gz 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77833.log 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36353.log.gz 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35754.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35754.log.gz 2024-04-25T01:12:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37388.log 2024-04-25T01:12:25.829 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34398.log 2024-04-25T01:12:25.830 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77833.log: /var/log/ceph/ceph-client.admin.37388.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77833.log.gz 2024-04-25T01:12:25.830 INFO:teuthology.orchestra.run.smithi033.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37388.log.gz 2024-04-25T01:12:25.830 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36197.log 2024-04-25T01:12:25.830 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34398.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34398.log.gz 2024-04-25T01:12:25.830 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36834.log 2024-04-25T01:12:25.831 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38822.log 2024-04-25T01:12:25.831 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36197.log: /var/log/ceph/ceph-client.admin.36834.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36197.log.gz 2024-04-25T01:12:25.832 INFO:teuthology.orchestra.run.smithi033.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36834.log.gz 2024-04-25T01:12:25.832 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34405.log 2024-04-25T01:12:25.832 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38822.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38822.log.gz 2024-04-25T01:12:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77937.log 2024-04-25T01:12:25.833 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77248.log 2024-04-25T01:12:25.834 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34405.log: /var/log/ceph/ceph-client.admin.77937.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34405.log.gz 2024-04-25T01:12:25.834 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77937.log.gz 2024-04-25T01:12:25.835 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77432.log 2024-04-25T01:12:25.835 INFO:teuthology.orchestra.run.smithi138.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T01:12:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77248.log.gz 2024-04-25T01:12:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36704.log 2024-04-25T01:12:25.836 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34195.log 2024-04-25T01:12:25.837 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77432.log: /var/log/ceph/ceph-client.admin.36704.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77432.log.gz 2024-04-25T01:12:25.837 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33591.log 2024-04-25T01:12:25.837 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36704.log.gz 2024-04-25T01:12:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.34195.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34195.log.gz 2024-04-25T01:12:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33883.log 2024-04-25T01:12:25.838 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33591.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33591.log.gz 2024-04-25T01:12:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33116.log 2024-04-25T01:12:25.839 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32834.log 2024-04-25T01:12:25.840 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.33883.log: /var/log/ceph/ceph-client.admin.33116.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33883.log.gz 2024-04-25T01:12:25.841 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35884.log 2024-04-25T01:12:25.841 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33116.log.gz 2024-04-25T01:12:25.841 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.32834.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32834.log.gz 2024-04-25T01:12:25.841 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33144.log 2024-04-25T01:12:25.842 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35001.log 2024-04-25T01:12:25.842 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35884.log: /var/log/ceph/ceph-client.admin.33144.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35884.log.gz 2024-04-25T01:12:25.843 INFO:teuthology.orchestra.run.smithi033.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38720.log 2024-04-25T01:12:25.843 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33144.log.gz 2024-04-25T01:12:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35001.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35001.log.gz 2024-04-25T01:12:25.843 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78197.log 2024-04-25T01:12:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77510.log 2024-04-25T01:12:25.844 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38720.log: /var/log/ceph/ceph-client.admin.78197.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38720.log.gz 2024-04-25T01:12:25.845 INFO:teuthology.orchestra.run.smithi033.stderr:gzip 88.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.78197.log.gz -- 2024-04-25T01:12:25.845 INFO:teuthology.orchestra.run.smithi033.stderr: /var/log/ceph/ceph-client.admin.38445.log 2024-04-25T01:12:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.77510.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77510.log.gz 2024-04-25T01:12:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78119.log 2024-04-25T01:12:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36301.log 2024-04-25T01:12:25.847 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38445.log: /var/log/ceph/ceph-client.admin.78119.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38445.log.gz 2024-04-25T01:12:25.847 INFO:teuthology.orchestra.run.smithi033.stderr: 89.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.78119.log.gz --verbose 2024-04-25T01:12:25.847 INFO:teuthology.orchestra.run.smithi033.stderr: -- /var/log/ceph/ceph-client.admin.37201.log 2024-04-25T01:12:25.848 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36301.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36301.log.gz 2024-04-25T01:12:25.849 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35832.log 2024-04-25T01:12:25.849 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35209.log 2024-04-25T01:12:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37201.log: /var/log/ceph/ceph-client.admin.35832.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37201.log.gz 2024-04-25T01:12:25.850 INFO:teuthology.orchestra.run.smithi033.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35832.log.gz 2024-04-25T01:12:25.850 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77633.log 2024-04-25T01:12:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38746.log 2024-04-25T01:12:25.851 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35209.log: /var/log/ceph/ceph-client.admin.77633.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35209.log.gz 2024-04-25T01:12:25.852 INFO:teuthology.orchestra.run.smithi033.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77633.log.gz 2024-04-25T01:12:25.852 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36145.log 2024-04-25T01:12:25.852 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.38746.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38746.log.gz 2024-04-25T01:12:25.852 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77989.log 2024-04-25T01:12:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37414.log 2024-04-25T01:12:25.853 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36145.log: /var/log/ceph/ceph-client.admin.77989.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36145.log.gz 2024-04-25T01:12:25.854 INFO:teuthology.orchestra.run.smithi033.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35806.log 2024-04-25T01:12:25.854 INFO:teuthology.orchestra.run.smithi033.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77989.log.gz 2024-04-25T01:12:25.854 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.37414.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37414.log.gz 2024-04-25T01:12:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37806.log 2024-04-25T01:12:25.855 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36249.log 2024-04-25T01:12:25.856 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.35806.log: /var/log/ceph/ceph-client.admin.37806.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35806.log.gz 2024-04-25T01:12:25.856 INFO:teuthology.orchestra.run.smithi033.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37806.log.gz 2024-04-25T01:12:25.857 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.admin.36249.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36249.log.gz 2024-04-25T01:12:25.873 INFO:teuthology.orchestra.run.smithi138.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T01:12:25.894 INFO:teuthology.orchestra.run.smithi033.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T01:12:25.937 INFO:teuthology.orchestra.run.smithi138.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T01:12:26.018 INFO:teuthology.orchestra.run.smithi033.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T01:12:26.399 INFO:teuthology.orchestra.run.smithi033.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T01:12:26.457 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T01:12:26.458 INFO:teuthology.orchestra.run.smithi033.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T01:12:26.460 INFO:teuthology.orchestra.run.smithi138.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T01:12:26.462 INFO:teuthology.orchestra.run.smithi138.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T01:12:26.462 INFO:teuthology.orchestra.run.smithi138.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T01:12:26.464 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-25T01:12:26.464 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m0.725s 2024-04-25T01:12:26.464 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m3.016s 2024-04-25T01:12:26.464 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.180s 2024-04-25T01:12:26.533 INFO:teuthology.orchestra.run.smithi033.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T01:12:26.547 INFO:teuthology.orchestra.run.smithi033.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T01:12:26.548 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T01:12:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m0.849s 2024-04-25T01:12:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:user 0m3.489s 2024-04-25T01:12:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m0.211s 2024-04-25T01:12:26.549 INFO:tasks.ceph:Archiving logs... 2024-04-25T01:12:26.549 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/remote/smithi033/log 2024-04-25T01:12:26.550 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:12:27.235 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/remote/smithi138/log 2024-04-25T01:12:27.236 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:12:27.608 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T01:12:27.618 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T01:12:27.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:12:27.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:12:27.686 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T01:12:27.696 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T01:12:27.696 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:12:27.699 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:12:27.709 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T01:12:27.710 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2024-04-25T01:12:27.710 INFO:teuthology.orchestra.run.smithi033.stdout:+hv01.front.sepi 44.4.53.2 2 u 26 64 377 0.080 -1.563 0.381 2024-04-25T01:12:27.710 INFO:teuthology.orchestra.run.smithi033.stdout:+hv02.front.sepi 63.231.80.2 3 u 23 64 377 0.076 -0.674 0.380 2024-04-25T01:12:27.710 INFO:teuthology.orchestra.run.smithi033.stdout:*hv03.front.sepi 74.6.168.72 3 u 27 64 377 0.083 -1.136 0.587 2024-04-25T01:12:27.710 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:12:27.711 INFO:teuthology.orchestra.run.smithi138.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T01:12:27.711 INFO:teuthology.orchestra.run.smithi138.stdout:============================================================================== 2024-04-25T01:12:27.712 INFO:teuthology.orchestra.run.smithi138.stdout:*hv01.front.sepi 44.4.53.2 2 u 31 64 377 0.128 -2.000 0.771 2024-04-25T01:12:27.712 INFO:teuthology.orchestra.run.smithi138.stdout:+hv02.front.sepi 63.231.80.2 3 u 26 64 377 0.082 +0.247 1.195 2024-04-25T01:12:27.712 INFO:teuthology.orchestra.run.smithi138.stdout:+hv03.front.sepi 74.6.168.72 3 u 26 64 377 0.108 -1.593 1.033 2024-04-25T01:12:27.712 INFO:teuthology.orchestra.run.smithi138.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T01:12:27.713 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T01:12:27.722 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T01:12:27.723 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T01:12:27.733 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T01:12:27.743 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T01:12:27.752 INFO:teuthology.task.internal:Duration was 925.984592 seconds 2024-04-25T01:12:27.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T01:12:27.762 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T01:12:27.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:12:27.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:12:27.802 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T01:12:27.802 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:12:27.802 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:12:27.853 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:12:27.853 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:12:27.864 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T01:12:27.864 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:12:27.897 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:12:27.930 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T01:12:27.931 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:12:27.965 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:12:28.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T01:12:28.039 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T01:12:28.039 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:12:28.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:12:28.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T01:12:28.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:12:28.097 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:12:28.106 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-04-25T01:12:28.120 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-25T01:12:28.131 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:12:28.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:12:28.164 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:12:28.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:12:28.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T01:12:28.188 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T01:12:28.189 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/remote/smithi033 2024-04-25T01:12:28.189 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:12:28.228 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672298/remote/smithi138 2024-04-25T01:12:28.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:12:28.249 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T01:12:28.250 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:12:28.268 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:12:28.298 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T01:12:28.309 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T01:12:28.309 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T01:12:28.320 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T01:12:28.320 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:12:28.322 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:12:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: 658673 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:12 /home/ubuntu/cephtest 2024-04-25T01:12:28.346 INFO:teuthology.orchestra.run.smithi138.stdout: 658462 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:12 /home/ubuntu/cephtest 2024-04-25T01:12:28.347 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T01:12:28.358 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T01:12:28.382 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/acls} duration: 925.9845921993256 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T01:12:28.382 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:12:28.470 INFO:teuthology.run:pass